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ABSTRACT 



The mathematical theory of optimal control/differential games is 
used to study the structure of optimal allocation policies for some tactical 
allocation problems with combat described by Lanchester-type equations of 
warfare* Both deterministic and stochastic attrition processes are con- 
sidered* For the optimal control of deterministic Lanchester-type attri- 
tion processes, a general solution algorithm for the synthesis of the 
optimal policy is developed. Optimal allocation policies are developed 
for numerous one-sided optimization problems of tactical interest in order 
to study the dependence of the structure of these optimal policies on model 
form. Consideration has been given to singular extremals, multiple extremals 
(including dispersal surfaces), and state variable inequality constraints. 

It is shown how to apply the theory of state variable inequality constraints 
to determine the optimal control of deterministic Lanchester-type processes 
in order to treat non-negativity restrictions on force levels and thus to 
study the dependence of optimal policies upon the force levels. Various 
attrition models are considered (reflecting different assumptions as to 
target acquisition process, command and control capabilities, target 
engagement process, variations in range capabilities of weapon systems). 
Solutions are developed for Lanchester-type equations of modern warfare 
with variable attrition-rate coefficients. The optimal control of the 
Lanchester stochastic process is studied. 

This task was supported by the Naval Analysis Programs, Office of 
Naval Research under ONR Project Order P02-0150 and Task Number NR 276-039. 

Prepared by: 



PREFACE 



This report documents my research on tactical allocation in 
dynamic combat situations over the past two years under the sponsor- 
ship of the Office of Naval Research (both as part of the Foundation 
Research Program at the Naval Postgraduate School and under contracts 
NR 276-027 and PO 2-0150). Some of this work is recent; some of it 
is not so recent. In all cases, however, the documentation is new 
and does not duplicate any past reports. 

Much of this work has been submitted for publication in the 
open literature. However, because of long n turn around times" for 
refereeing in several journals (sometimes over a year before receiving 
the referees' initial comments), I felt that it was appropriate to 
include several manuscripts as appendices that had been previously 
submitted for publication in the open literature. Hence, some of 
this work is not of recent vintage: the paper "On the Solution to 

Lanchester-Type Equations With Variable Coefficients" was submitted 
for publication to Operations Research in January 1971. In many 
instances I have obtained far more extensive results but have not had 
the time to fully document them. 

Thus, the appendices of this report are more or less separate 
entities in themselves and may be read independently of each other. 
Accordingly, each appendix contains its own list of references and 
reference numbers apply only within the appendix in which they appear. 



A similar remark applies to the numbering of equations. Also, the 
pagination is separate for each individual appendix. 

The emphasis in this report is on the applications of control 
theory to tactical allocation problems in dynamic combat situations. 
Although consideration of the various problems contained in this 
report has required knowledge of research on the frontiers of deter- 
ministic optimal control theory, I have felt it to be inappropriate 
for the scope of this research to supply proofs for any optimization 
theory results. However, I have tried to give the appropriate 
references to support all statements of theory contained herein. 
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1. Introduction. 



This report documents research performed under the sponsorship 
of the Office of Naval Research (contract PO 2-0150) during the time 
period of 27 March 1972 to 16 June 1972. We discuss the results of 
a study of some idealized models for the optimization of combat 
dynamics and their possible implications for defense planners. Our 
research approach has been to combine Lanchester-type formulations 
of combat attrition (both deterministic and stochastic) and generalized 
control theory [64] (both deterministic and stochastic optimal control, 
dynamic programming, differential games) . 

A quantitative theory of tactical allocation is developed 
through the examination of a sequence of simplified models. These 
combat models are too simple to be taken literally but should be 
interpreted as indicating general principles to serve as hypotheses 
for subsequent computer simulation studies or field experimentation. 

The effects of various modelling assumptions upon the structure of 
the optimal allocation policies are systematically studied by con- 
trasting the solutions for various models. 

A major result of our research is that optimal tactical 
allocation policies are quite sensitive to the precise type of model 
adopted, even as to whether the tactical scenario lasts a specified 
period of time or terminates only when a pre-determined state has 
been reached. Insights are provided into such important questions 



as : 
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(1) How should targets be selected? 

(2) Do target priorities change with time? 

(3) Do battle termination circumstances affect the optimal 
allocation policies? 

(4) How does the nature of the attrition process affect target 
selection? 

(5) What is the effect of ammunition constraints? 

(6) How does the uncertainty and confusion of combat affect 
the optimal selection rules? 

To develop our theory of tectical allocation we have extensively 
studied some specific combat scenarios. Optimal tactics for the 
following have been studied: selection of target type at which to 

fire, regulation of firing rate. The influences of the following 
factors have all been considered: 

(1) combatant objectives (form of criterion function and 
valuation of surviving forces) , 

(2) weapon system performance characteristics, 

(3) termination conditions of conflict, 

(4) force strengths, 

(5) type of attrition process, 

(6) effect of resource constraints, 

(7) range capabilities of weapon systems. 

The tactical situations are described by deterministic Lanchester- 
type equations of warfare. The combat continues over a period of time 
with a choice of tactics available to both sides and subject to change 
over time. The mathematical theory of deterministic optimal control/ 
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differential games has been used to solve the problems under 
consideration. Thus, it seems appropriate to discuss these techniques 
briefly. 

a. Differential Games/Optimal Control . 

The theories of optimal control and differential games were 
developed for optimization problems in which the (deterministic) 
system* s dynamics are described by a system of ordinary differential 
equations. The reader can find references additional to those given 
here and a brief review of past developments in our previous report 
[113]. 

In a two-person zero-sum deterministic differential game (hence- 
forth abbreviated to simply differential game) each player chooses 
strategies (for precise definitions, see [12] or [52]) in order to 
maximize his own criterion functional (which when added to that of 
his opponent yields zero) for a system whose dynamics is governed by 
a system of ordinary differential equations. An example problem 
relevant to our research on tactical allocation problems is 



n m 

maximize minimizej £ w.y.(T) - £ v.x.(T)} with T specified, 

hj (t) hj (t) i=1 11 1=1 1 1 

dx . n 

subject to : — — = r . - j ip . .a . .y . for i = 1 

dt 1 ^ J 



dy . m 

“TT " 1 = S . - 7 <J> . .b . .X . 

dt 1 j=l 1J 1J J 



for i = 1 , . . . ,n, 
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with 



where 



m 

l 'l'. H = 1 for j = 1, . . . ,n, 
i=l J 

n 

I <J > = 1 for j = 1,. . . ,m, 
i=l 3 



^0 and x # ,y # ^ 0, 
ij ij i i 



( 1 ) 



and 



>y^(t) are force levels, 
are replacement rates, 

V i ,W i are t * ie u tmtles assigned survivors, 



a ij 


is 


the 


rate at which one Y . 

J 


unit 


can destroy 


X. 

1 


b. . 
ij 


is 


the 


rate at which one X. 

3 


unit 


can destroy 


Y. 

l 


'ij 


is 


the 


fraction of X. 

J 


who 


fire at 


V 






is 


the 


fraction of Y. 

J 


who 


fire at 


V 





For our illustrative purposes here, let us assume that the battle is 
scheduled to last a prescribed duration of time (we further assume that 
no side is annihilated before T) . Such problems (mainly pursuit and 
evasion) have been extensively studied in recent years. The goal of 
this research has been to study tactical allocation problems using 
such models. 

Pioneering research on differential games was done by R. Isaacs 
and culminated in his book [72] . Isaacs developed a general principle 
(the tenet of transition) in order to develop his "main equation" 
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(which is basically a two-sided extension of the Hamilton-Jacobi 
equation (see [52])). L. Berkovitz [12] has justified in a rigorous 
manner many of Isaacs 1 original results, which were heuristically 
obtained. In [10] Berkovitz developed many basic results for differ- 
ential games according to variational arguments. 

The above theories apply to differential games which possess 

solutions in pure strategies. It is a simple matter to devise problems 

for which this is not true (see [11], [13], [52]). However, Isaacs 

conjectured that when the Hamiltonian is separable, i.e. a function 

independent of <j> plus a function independent of ip , then the 

* 

differential game has a solution in pure strategies. This conjecture 
was later proven to be true by L. Berkovitz (see pp. 170-173 of [10]) 
and A. Friedman [52] . We shall see below that the Hamiltonian for 
(1) will be separable in this sense. 

Let us consider the above example (1) again. The Hamiltonian 
is given by 



m 



H(t,x,p,<M) = l p.{r - l l a y ) 
i=l j=l J J J 



m 



+ 1 - I *..b..x.}, 

±=i 1 1 j=i j 



( 2 ) 



where 



p_^(t) denotes the dual variable corresponding to x_^. 



and q ± (t) denotes the dual variable corresponding to y^ . 



We use the symbol <p to denote all the variables under control of X. 
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In the case when the differential game has a saddle-point (in pure 
strategy solutions) so that a value exists, let us denote the value 
function as W(t,x,y). This value function is well-known [12] to be 
C* (i.e. continuously differentiable with respect to each of its 
arguments) except at certain manifolds of discontinuity. Except at 
such manifolds, one can make the identification 



* 

P ± (t) 




(t), 



and 



q i( t) -|S-(t). 

From (2) , we see Hamiltonian is indeed separable in the sense discussed 

above. However, as not noted by Isaacs, Berkovitz, or Friedman, the 

above expression (2) for the Hamiltonian function is appropriate only 

when none of the state variable inequality constraints (SVIC’s) of (1) 

* 



is active. In Appendix E, we discuss the fact that although problems 
with SVIC's have been extensively studied in the literature of deter- 
ministic optimal control theory (our study of tactical allocation 
problems in the Lanchester theory of combat, however, has led to the 
uncovering of an important gap in the existing theory of SVIC f s [115]), 
a current gap in the theory of differential games is the lack of treat- 
ment of problems (such as (1) above) with inequality constraints on 
functions of the state variables only (the control variables not 



In (1) x_^ for i = 1 , . . . ,m and y . for j = l,...,n are state 

variables , 'Stfhile the <}> ? s and i|; f s sire called control variables. An 
example of a SVIC is the condition that x^ ^ 0, i.e. the X]_ force 
level must be non-negative. Thus, (2) is appropriate only for x^ > 0 
and y ± > 0 for all i. 



7 



explicitly appearing in these inequality constraints). In fact, the 
approach that R. Isaacs used to solve the "War of Attrition and Attack" 
(see pp. 96-104 in [72]) is easily shown to be inadequate for other 
similar problems (for further details see Appendix E) . (This important 
gap in existing theory and the inadequacy of Isaacs 1 (and others) 
treatment of SVIC f s is not noted in the recent work of S. Sternberg 
[ 111 ].) 

Thus, when x_^ > 0 and y_^ > 0 for all i, the Hamiltonian 
is given by (2) . Hence the Hamiltonian is separable (in the sense 
discussed above) so that a solution in pure strategies can be found 
to (1) . Let us pursue further details of solution development to 
illustrate additional difficulties. 

Assuming that x^ > 0 and y > 0 for all i, we determine 

* * 

an extremal strategy pair, denoted as (<j> ,ip ) , by the max-min 
principle 



* * * * 

Maximum minimum H(t ,x,p ,ij>) * H(t,x,p ,ip ) . 



(3) 



ip ■ ■ 
ij 



♦u 



This leads to the problem 



* 

maximize! £ 



m 

subject to : £ iK . = 1 

i=l J 



^ 0 for j = 



(4) 
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which may be routinely solved to yield 



where 






1 for i = j , 



( 5 ) 



i.J 



0 otherwise. 



and k(j,t) is the index such that 



a k4 ( -p k^ = maximum { a li C-Pj^) » • • • » a mi (-P m )} . for j = 1 , • • • ,n. (6) 



Similarly we obtain 



n 



minimize{ £ <(> ± - b ii q ± } 
i=l J J 



n 



subject to : £ <J> . . = 1 



i-i 1J 



♦y iO for j = 1, . . . , m, (7) 

* 

so that <j>„(t) is analogously determined. 

It may be that the index k(j,t) is not unique, i.e. the 
linear program (4) has alternate optima. This causes no difficulty 
unless this situation continues for a finite interval of time. When 
this happens, we say that the corresponding segment of the battle 
trajectory is a singular subarc . Of great practical significance is 
the fact that the max-min principle does not provide adequate tests 
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for whether a singular subarc can yield an optimal outcome. Thus, 
the possibility of singular subarcs must be investigated in tactical 
allocation differential games, since the control (decision) variables 
appear linearly in problems such as (1) . We shall discuss this 
important aspect further below (see Appendix D for further details in 
one-sided case) , but for now let us simply state that it may be shown 
that it is not possible to have a singular solution to (1) . 

From (5) we see that this model (1) yields that the optimal 
tactic is to concentrate all fire on one enemy target type. However, 
we really don’t gain that much insight into changes in the optimal 
target selection policy with time, since we don’t easily see explicitly 
how the dual variables (and hence the optimal tactic via (4) and (7)) 
change over time. The adjoint system of differential equations for 
the dual variables is, of course, given by 



* 




with 

P*(t-T) - -V. 

V t=T > -“r 
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Hence, at the end of battle at t = T we have 



hj (t T) 6 i,k(j,T)’ 



(9) 



where k(j,T) is the index such that 



akj v k = maximum{a v^.^.a v^}, for j = 1 



and 



T) h>£(j ,T) * 



( 10 ) 



where £(j,T) is the index such that 

b rt ,w n = maximum{b- ,w- , . . . ,b .w } , for j = 1 , . . . ,m. 
l lj 1 nj n J 

In Appendix C, we show how the extremal (candidate for optimal 
control) control may be explicitly determined as a function of time 
for a one-sided analogue of the problem at hand. 

Thus, we see that for separable controls as occur in target 
selection problems in the Lanchester theory of combat the solution 
method (i.e. the max-min principle) essentially (speaking somewhat 
imprecisely now) reduces the problem of determining the solution of the 
differential game to solving two "separated" one-sided optimization (optima 
control or variational) problems. Moreover, in [10] L. Berkovitz 
reduces a differential game to two such control problems (problems 
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of Bolza with differential inequalities as side conditions) in order 

* 

to deduce necessary conditions for optimal paths. 

It should therefore be clear that if there are difficulties 
(either theoretical or computational) in solving such optimal control 
problems, then there will be difficulties in solving the corresponding 
differential game. (Alternatively, we may think of an optimal control 
problem as a differential game in which the strategy(s) of one of the 
two players is fixed beforehand.) Accordingly, our research approach 
has been to study optimal target selection problems for dynamic combat 
situations with decisions available to both sides by first considering 
corresponding one-sided (optimal control) problems. After the difficult 
points have been mastered in the simple one-sided problems, then the 
differential games are most profitably tackled. 

In the next section we discuss some aspects of deterministic 
optimal control problems that can cause difficulties. All these 
subtle points may also arise in differential games. We have found 
that by studying both the theory and examples of difficult aspects in 
one-sided problems we have appreciably increased our ability to solve 
corresponding differential games. In fact, as noted above, one of our 
research findings is that a theory of state variable inequality con- 
straints (which is essential in problems such as (1) because of the 



This is done by the famous method of Valentine (see [125]). M. Hestenes 
[61] formulated the (now standard) general control problem and deduced 
necessary conditions (including the maximum principle (frequently 
referred to as the Pontryagin maximum principle)) according to this pro- 
gram in 1949. The same approach is readily extended to nonlinear program- 
ming (including linear programming) , but this apparently has been over- 
looked by workers in the field. However, our recent paper [118] remedies 
this gap. 
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requirement that force levels be non-negative) is lacking in the 

* 

current theories for differential games. We plan to develop such a 
theory in the future. 

Based on this intimate relationship between the mathematical 
theories of differential games and optimal control [63], our research 
approach (as stated above) has been to consider one-sided versions of 
some tactical allocation structures before tackling the more realistic 
two-sided decision problem. Our intent has been to make sure that 
both theoretical and computational aspects are firmly established for 
these optimal control problems before attempting to solve the more 
complex differential game versions of such problems. 

Optimal control theory has emerged as a vigorous discipline 
since World War II. Professor Magnus Hestenes of the USA apparently 
first gave the now standard control formulation for a variational 
problem (as well as developed optimality conditions) and urged others 
to follow his approach in 1949 [61]. Professor Hestenes did not feel 
that it was appropriate to publish this work in the open literature, 
since, for example, his development of a "Weierstrass maximum principle” 
was a translation of well-known results from the classical calculus 
of variations. Consequently, this pioneering work [61] received little 



A. Friedman (see Chapter 6 of [52]) has developed a non-variational 
theory of differential games via a limit approach. Although he has 
established that a certain class of differential games (our problem. (1) 
belongs to this class) with state variable inequality constraints (SVIC's) 
do possess a value and the optimal strategies (see pp. 238-240 of [52]) 
does not appear to be adequate. Specifically, Friedman makes an unsup- 
ported assumption about a property of optimal strategies (the assumption 
that it is optimal to have x^(T) > 0) . 
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attention from researchers other than some at RAND. In 1956 Pontryagin 
and his associates announced their celebrated "maximum principle" [19] . 
This event ushered in over a decade of intense activity on variational 
methods. Due to this intense activity in the field of optimal control 
theory during the past 15 years and our specific interest in applica- 
tions to military operations research problems, it does not seem 
appropriate to cite here voluminous research articles (however, see 
[3] and [99] for surveys). The current state of the art in deterministic 
optimal control theory is reflected in the following contemporary text- 
books and monographs from which a further more specialized guide to 
the literature can be obtained [4], [6], [18], [28], [40], [42], [45], 
[51], [62], [89], [91], [101]. 

b . Special Features of Optimal Control Problem . 

As we have discussed above, our research approach has been to 
initially study dynamic tactical allocation problems by considering 
one-sided optimal control versions of them. Moreover, these deter- 
ministic optimal control problems that we have studied within the 
framework of the Lanchester theory of combat have contained certain 
special features that have caused difficulty (both theoretically and 
computationally) . Let us consider some examples to illustrate such 
difficulties . 

For illustrative purposes, we first consider the following 
optimal control problem for target selection in Lanchester combat 
(see [117] (reproduced in this report as Appendix A) for further dis- 



cussion of model) 
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maximize{ry(T) - px (T) - qx (T) } with T unspecified, 

<P(t) 1 

dx i 

subject to: -j-^— = -<j>a^y 



dx 2 

Tt (i-<|> )a 2 y 



iz = 

dt 



- b i x i 



b 2 X 2 



(ID 



0 £ <J> £ 1 (inequality constraint on control variable) , 



x^ y x^ 9 y ^ 0 (state variable inequality constraints), 

where 

x^(t) , x^(t), y(t) are force levels, 
p, q, r are utilities assigned survivors, 

a^, a^y b^, b^ are (constant) attrition-rate coefficients, 
and <p is the fraction of Y fire directed at X^. 



The stopping rule for the battle is that the conflict terminates 
at t = T defined by 

(a) y(T) = 0, 

or (b) x 1 (T) = x 2 (T) = 0. 



Further analysis yields that there are five cases which must be 



considered : 



15 




x 1 (T) 
x 1 (T) 
x 1 (T) 
x 1 (T) 
x x (T) > 0, 



x 2 (T) > 0, 
x 2 (T) = 0, 
x 2 (T) = 0, 
x 2 (T) = 0, 
x 2 (T) > 0, 



= o, 

= 0 before 
= 0 after 

> o. 



y(T) = o, 
y(T) > o, 
y(T) > 0, 
y(T) = 0, 
y(T) = 0. 



A constraint such as x^ ^ 0 is referred to as a state 
variable inequality constraint (SVIC from [29]). When such a constraint 
is active (as when x^(t) = 0 for t^ £ t £ t.^) , the usual necessary 
conditions of optimality (maximum principle) require modification: 
the adjoint equations are modified, boundary conditions for the dual 
variables require special treatment, juncture (corner) conditions 
must be satisfied, some special multiplier conditions must be satisfied 
(see Appendix E for details) . This area of deterministic optimal 
control theory (problems with SVIC's) has had much recent research 
activity (see bibliographies in [79], [115]). However, the appropriate 
treatment of a SVIC in order to determine the optimal trajectory is 
apparently not widely known among practitioners of optimal control 
(as pointed out by our recent note [115]), although the appropriate 
necessary conditions (as well as sufficient conditions) are, of course, 
well-known by active researchers . 

When x^,x 2 ,y > 0, the Hamiltonian for the problem (11) is 



given by 
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H(t,x,p,<j>) = -P 1 (t){<j>a 1 y} - p 2 (t){ (l-<|))a 2 y} - p^tHb^ + b^}, (12) 

where p^(t) is the dual variable corresponding to x^, etc. By 
(12) we see that the Hamiltonian is a linear function of the control 
variable so that a singular solution [80], [81] is possible. Singular 
solutions to target selection problems in the Lanchester theory of 
combat are discussed in Appendix D. This aspect of the optimal 
control of deterministic Lanchester attrition processes is of great 
practical significance, since it means that it is possible to have 

•k 

other than <j> = 0 or 1 as an optimal fire distribution policy (in 

contradiction to the results presented in [111]). 

Let us further elaborate upon singular solutions in deterministic 
problems of optimal control. In such a problem, the maximum principle 
may fail to determine an optimal trajectory, since the maximization 
of the Hamiltonian may not lead to a well-defined expression for 
optimal control [80], [81] (also see Chapter 8 of [28]). Singular 
solutions usually occur when the Hamiltonian is a linear function of 
the control variables (as is the case for fire distribution problems 
in the Lanchester theory of combat as formulated by Isbell and Marlow 
[74] and Weiss (see pp . 94-95 of [129])). However, all problems for 
which the Hamiltonian is a linear function of the control variables 
do not have singular subarcs (see below) in their solution. In par- 
ticular, we shall see that problem (11) does not. 

The above problem (11) has one control variable (denoted as <}>) , 
and it appears linearly in the Hamiltonian. By a singular subarc we 
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denote that part of an optimal trajectory on which the maximum 
principle cannot be used to determine the control because the coeffi- 
cient of the control variable in the Hamiltonian is zero (see pp . 226- 
227 of [80]). Then the term "singular solution 11 will be used to 
refer to any optimal trajectory which contains one or more singular 
subarcs . 

To elaborate further, when the Hamiltonian H is a linear 

3H 

function of the control variable <j> , then if — = 0 for a finite 

09 

interval of time (or, another way to say this, the coefficient of 
(j> vanishes identically for a finite interval of time) , then the 
maximum principle does not determine the control. When this situa- 
tion occurs, neither the maximum principle nor the classical variational 
theory provide adequate tests for maximality of the singular subarc. 
However, Kelly [82], [83] generalized the Legendre-Clebsch condition 
of local optimality of extremal paths containing singular subarcs 

(see [75], [78], [110] for very recent developments). 

3H 

Observe that on a singular subarc for which — = 0 for a 

O9 

finite interval of time all feasible values of <j> maximize the 

Hamiltonian. On a singular subarc we determine the singular control 

8H 

by requiring that we remain on the singular subarc, i.e. remains 

3H 

zero. If — is to be identically equal to zero for a finite 

d<f> 

interval of time, then all of its derivatives with respect to time 
must also be equal to zero. We determine the singular control, which 
keeps the system on the singular subarc, by considering as many of the 
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time derivatives of — as are required for the control variable 

to appear explicitly so that it may be determined from an algebraic 
equation. Thus, in general we consider 



JH = _d/9ih = d 2 
9<j> d t ^ 9 dt 2 ^' 



(13) 



We must further check to see that we can get a maximum return 
(in the case when we wish to maximize the criterion functional) from 
use of the candidate singular subarc. The following condition (gener- 
alized Legendre-Clebsch condition) is necessary for a singular subarc 
to yield a maximum return 

2k 

< 14 > 

It is obtained by examining the negative semidefiniteness of the 
second variation for a special class of explicitly defined control 
variations [83]. For the problem (15) considered below, it suffices 
to consider the generalized Legendre-Clebsch condition with k = 1. 

For fire distribution problems in the Lanchester theory of com- 
bat it turns out that when the rate of target-type attrition is pro- 
portional to the number of enemy firers only (as in the above problem 
(11)), it is impossible (excluding the pathological case when a^b^ = 

a 0 b 0 ) to have — f-rr) = 0. Thus, it is impossible to have a singular 
L 2 dt oy 

* 

solution. (However, such important aspects were not discussed in [111] ). 

: k 

This is a common gap in applications of optimal control theory by many 
practitioners. For example, the vital subject of singular solutions is 
not discussed in [1], [2], [67], or [106], even though singular solutions 
are treated by heuristic means. 
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Our research has yielded the fact that the possibility of 
singular solutions must be investigated for all such fire distribution 
problems in the Lanchester theory of combat* In particular, a singular 
solution arises in the following problem 



maximize{ ry (T ) - px (T) - qx 9 (T)} with T specified, 
♦ (t) 

dx i 

subject to: = -ij>a^x^y 



dx 



2 



dt 



-(l-4>) a 2 x 2 y 



ix = 

dt 



'Vi 



b 2*2 



(15) 



0 £ <J> £ 1 and x ,x 2 ,y it 0. 



In Appendix D we show that it is possible to have an optimal fire 

distribution policy that is not an extreme point in the control 

* 

variable space, e.g. <f> = a^/ (a-j+a^) * For t * ie P r °bl em at hand, this 

explicit expression for the singular control is obtained from the 
equation 





d 3H 

by use of the conditions — = — f— 1 = 0 and the canonical equations 

d(J) dt^dqr 

(i.e. both state and adjoint system). 

A more subtle difficulty in obtaining a complete solution to 
fire distribution problems is that there may be more than one extremal 
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(an extremal is a path on which the necessary conditions of optimality 
(maximum principle) are almost everywhere satisfied) leading from an 
initial point in the force level (state) space to the terminal surface. 
This occurs both in the Isbell-Marlow problem [74], [117] and the 
supporting weapon system game of H. K. Weiss [130], [122]. Such a 
situation occurs when it is possible to reach different points on 
the terminal surface from the same point in the initial state space 
(see pp. 276-284 of [4]). Such extremals are difficult to identify, 
since sufficient conditions of optimality [53] may be satisfied along 
each of the multiple extremals. 

In problem (11) above, there are circumstances under which more 
than one extremal (battle trajectory on which the necessary conditions 
are satisfied) leads from the same initial force levels x°, x° , yQ 
to the terminal surface. For example, under the appropriate circum- 

k 

stances extremals may lead to both and C^. Moreover, it may 

be shown that sufficient conditions of optimality [53] are satisfied 
for each extremal. However, this difficulty in solution development 
is probably best treated by example so that we defer further in depth 
discussion until the appendices. 

To summarize, we have identified three aspects of fire distri- 
bution problems in the Lanchester theory of combat that have not been 
adequately treated previously. These aspects are as follows: 

•k 

Actually, there are two types of extremals that may lead to . 
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(1) state variable inequality constraints, 

(2) singular subarcs, 

(3) multiple extremals. 

Moreover, each of these aspects can be vital in developing solutions 
in applications. 

2 . Objectives of Research . 

The general objective of this research is to determine the 
structure of the optimal fire distribution/ target selection policies 
for various specific scenarios of tactical interest. The specific 
objective of this research on applications of control theory to Lan- 
chester-type formulations of combat attrition is to determine the 
sensitivity of target selection and force allocation strategies to 
the form of the attrition model and to the time history of force 
levels . 

3. Related Work of Others . 

Additionally, we will review in more depth developments in the 
quantitative analysis of military tactics/strategies through Lanchester- 
type models of warfare (i.e. variational models to determine optimal 
allocations (e.g. target selection, fire distribution, etc.) as a 
function of time and/or force levels in combat between heterogeneous 
forces) . Hence, we consider the related work of others in the fields 



of 
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(a) Lanchester-type models of warfare, 

(b) differential games, and 

(c) optimal control of Lanchester attrition processes. 

We are aware of probably well over 200 titles of papers and 
books which contain information that, in one sense or another, would 
probably assist in the attainment of the above stated objectives of 
this research. For the present we shall highlight some major works. 

In this survey we consider the subject area of differential games to 
be representative of applicable material within the more extensive 
framework of generalized control theory" (see [64], where this term 
was coined by Y. C. Ho). More extensive reviews of pertinent litera- 
ture are to be found in two of our past reports [112], [113]. 

a . Lanchester-Type Models of Warfare . 

One of the earliest attempts to establish a mathematical model 
of the dynamics of mass combat was by Lanchester [87] in 1916. He 
postulated several deterministic models that were a system of ordinary 
differential equations which related the strengths of opposing military 
forces to length of combat. During World War II B. Koopman extended 
Lanchester ! s results and also suggested a reformulation of the problem 
in stochastic form [96] . After World War II the RAND Corporation 
carried on further studies [60], [97], [98] whose results were 
summarized by R. Snow [109]. H. K. Weiss (then at Aberdeen Proving 
Ground) and others [5], [26], [46], [58], [59], [128], [129] have 
subsequently extended deterministic Lanchester models. 
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R. Brown developed models for the stochastic analysis of combat 
[27]. The relationship between the above mentioned stochastic and 
deterministic Lanchester formulations was pointed out relatively 
early in their development (see [109], for example) but is probably 
best discussed in a recent paper by B. 0. Koopman [86]. The first 
probability analysis of Lanchester-type equations apparently appeared 
in the (now classical) Morse and Kimball book [96]. Using a stochastic 
model, an expression was obtained for the probability of one side 
winning in a fight to the finish in which the random attrition process 
corresponds to a deterministic Lanchester linear-law attrition process. 
Stochastic and deterministic model results are compared. A more in 
depth comparison of stochastic and deterministic model results has 
been given by G. Weiss [127]. R. Brown developed not only expressions 
for the probability of winning but also approximations to the probability 
of winning as determined by stochastic models of combat attrition 
corresponding to both the Lanchester linear law and the square law 
[27]. Further extensions of these results have been given by Smith 
[108] and Kisi and Hirose [85]. To date, the most thorough comparison 
of deterministic and stochastic homogeneous-force Lanchester formula- 
tions has been by G. Clark (see Chapter 11 of [16] and [41]), who has 
also developed the time-state solution for the stochastic attrition 
process corresponding to a deterministic Lanchester linear-law process 
[41]. 
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S. Bonder [22] did the pioneering work on the determining the 
Lanchester attrition-rate coefficient (for weapon systems that adjust 
fire based on the results of the immediately preceding round). C. 
Barfoot [7] later suggested defining the Lanchester attrition-rate 
coefficient as the harmonic mean of the attrition rates (or, equiva- 
lently, the reciprocal of the expected time to kill a single target) . 
Barfoot also presented an alternate and more general method for com- 
puting the Lanchester attrition-rate coefficient. Bonder [23] then 
showed that the mean attrition rate for Markov-fire weapons is readily 
obtainable by the methods described in [22] when one defines the 
Lanchester attrition-rate coefficient as the reciprocal of the expected 
time to kill a single target. Kimbleton [85] has extended the above 
work by deriving the distribution of the time required to destroy a 
target. Other papers [104], [103] have discussed the estimation of 
parameters for the Lanchester attrition-rate coefficient. 

A good review of the Lanchester theory of combat is by Dolansky 
[46] , and this includes a comprehensive list of references through 
1965. A recent thesis directed by us [56] contains a fairly compre- 
hensive bibliography of articles appearing in the open literature 
since 1964. Recent trends in the Lanchester theory of combat were 
discussed at a 1967 NATO Conference [48], [126]. Of particular rele- 
vance to our recent work is the pioneering work of D. Etter [49] (also 
to appear in [48]), who has formulated and (partially) solved several 
allocation of fire problems (see below for further details) . Etter 
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has also given consideration to the spatial dimension of combat and 
force mobility [49]. 

H. K. Weiss [129] extended Lanchester-type equations to include 
the relative movement of two homogeneous forces, allowing time and 
space to be u traded M for casualties. He considered the two attrition- 
rate coefficients to be dependent upon force separation in such a way 
that their ratio was constant. S. Bonder [20], [21] used Weiss 1 extension 
to study the effects of mobility and various range dependencies for 
the attrition-rate coefficients on the number of surviving forces. He 
has studied scenarios for combat between two homogeneous forces, and 
developed solutions for the case of a constant attack velocity under 
special circumstances. In our past research, we have shown [114] that 
a simple closed-form solution is possible in all those cases when the 
ratio of attrition-rate coefficients is constant, regardless of whether the 
attack velocity is constant or not (see also [116]). We have subse- 
quently developed [123] a completely general solution to variable 
coefficient Lanchester-type equations for combat between two homogeneous 
forces and further developed analytic results in the following important 
cases: (1) opposing weapon systems whose kill rates have different 

range dependencies and (2) weapons systems with different effective 
ranges. Some related work appears in [24] and [43]. 

b . Differential Games . 

The study of differential games was initiated by R. Isaacs at 
RAND in the early 1950 f s [68], [69], [70], [71], but his work has not 
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been available to a wide audience until more recently [72]. His basic 
concept, "the tenet of transition," is generalization of Bellman*s 
[8] "Principle of Optimality" to a competitive environment, and this 
is used to develop necessary conditions for optimal strategies. A 
more recent and more rigorous development of these basic necessary 
conditions is by Berkovitz [12]. 

Since the excellent paper by Ho, Bryson, and Baron [65] in 1965, 
there has been a literal explosion of papers on differential games but 
almost all deal exclusively with pursuit-evasion problems. Excellent 
survey papers which bear this out are by Simakova (Russian literature) 
[107] and Berkovitz [13]. (However, applications to the Lanchester 
theory of combat are discussed in the next section.) Besides Isaacs* 
book, others which treat differential games are by Blaquiere et al 
[17] (extension of their geometric approach to optimal control), 

Bryson and Ho [28] (Chapter 9), and Friedman [52]. This latter work 
appeals to be the most authoritative one on the mathematical theory 
of differential games to date and contains a fairly comprehensive 
bibliography for the purely mathematical aspects of differential 
games . 

c . Optimal Control of Lanchester Attrition Processes . 

In 1964 Dolansky [46] noted that an underdeveloped area in the 
Lanchester theory of combat was optimal target selection in combat 
between heterogeneous forces (optimal control/differential games). 

Our past results [117], [122] have extended all the previously published 
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results [74], [130] in the open literature cited by Dolansky. Recently, 
we have extended these results even further [119], [120], [121]. 

In [117] we showed that the solution originally obtained by 
Isbell and Marlow [74] (also given in [111]) is incorrect for a cer- 
tain range of model parameters and presented a general methodology for 
solving all such problems. In [122] we extended this methodology to 
differential games by considering the supporting weapon system game 
of H. K. Weiss [130], which was previously only treated by heuristic 
means. (It was not noted in [111] that Weiss did not apply the 
mathematical theory of differential games to obtain his solution to 
this problem.) 

More recently, we have discussed the influence of various 
factors (combatant objectives, termination conditions of conflict, 
type of attrition process, variable attrition-rate coefficients, and 
limited ammunition) [119]. In [120] we presented some preliminary 
results for optimal fire distribution/ target selection in combat 
between heterogeneous forces (the special case of a homogeneous 
force against heterogeneous enemy forces) . We considered both con- 
stant attrition-rate coefficients and some special cases of variable 
attrition-rate coefficients. In [121] we treated a simple problem of 
target selection for a homogeneous force in Lanchester combat against 
two enemy force types each of which undergoes attrition at a rate 
proportional to the product of the numbers of firers and targets. 
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Some similar problems had been previously studied by D. Etter 
[49] (also to appear in [48]). Although not using modern optimal 
control theory, his arguments (which failed to give consideration to 
the long-run battle outcome) did lead to results which in many cases 
anticipated many of our recent results. Etter [49] pointed out the 
fundamental difference between the optimal allocation of fire with 
square-law attrition and with linear-law attrition and has done some 
computational studies . 

Work in this area has also been done at the University of 
Michigan [24]. However, the dependence of optimal strategies upon 
model form (i.e. type of attrition process, battle termination condi- 
tions, replacements, combatant objectives, range dependent attrition 
rates, etc.) was not noted (our results are reported in Appendix C of 
[113] and [117]) and certain subtleties of optimal target selection 
policies not detected. Moreover, in [111] S. Sternberg apparently 
gave the same solution to a two-on-one problem as originally obtained 
by Isbell and Marlow [74] and presented the same solution development 
methodology as they did. In our past research [117] (reproduced for 
the reader’s convenience in this report as Appendix A (see also 
Appendix F)) we have shown that the solution originally obtained by 
Isbell and Marlow [74] is not completely correct. Hence, Sternberg’s 
claim (see p. 153 of [111]) that a "two-on-one example was then solved 
in its entirety” does not seem justified to us. 
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In view of the similarities in the research areas considered 
by Sternberg [111] and the research results presented in report, it 
seems appropriate to make a few critical comments about Sternberg ! s 
work. As we have noted above, Sternberg did not develop solution 
methodology capable of obtaining a complete solution to the two-versus- 
one fire distribution problem. Even more serious, however, is his 
failure to note the well-known fact [63], [65] that differential 
games are a class of two-sided optimal control problems. It is not 
surprising then that Sternberg does not discuss the well-known control 
theory results for singular solutions, state variable inequality con- 
straints (SVIC’s), non-uniqueness of extremals, etc., as applied to 
differential games in the Lanchester theory of combat. Furthermore 
Sternberg [111] does not note that a theory for state variable 
inequality constraints has not been developed for differential games 
or that Weiss [130] did not use the necessary conditions of optimality 
to develop his M solution" to the supporting weapon system game. 

The practical significance of the above facts is that, for 
example, the solution of the war of attrition and attack given by 
Isaacs in [72] (and partially reproduced by Sternberg on pp. 17-25 of 
[111]) is not justified (the treatment of the boundary conditions for 
the dual variables is inadequate as is that for constrained subarcs 
when a SVIC is active) . (It is easy to give an example (see Appendix 
E) for which Isaacs 1 approach does not yield the correct solution.) 
Hence, Sternberg’s attempt to solve a M two-on-two M differential game 
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appears premature to us. Our own research indicates that use of the 
appropriate necessary conditions of optimality appreciably reduces 
the volume of allocation strategy combinations that have to be 
considered . 

Additionally, Sternberg [111] did not investigate the dependence 
of optimal allocation strategies upon model form. His major research 
conclusion (on pp. 153-154 of [111]) that optimal allocation strategies 
in Lanchester-type processes are always 0,1 is incorrect as stated, 
since it is easily shown that for two-versus-one combat in which 
target types undergo attrition at rates proportional to the product of 
the numbers of firers and targets, the optimal allocation strategy may 
be other than 0 or 1 (see [121]). Thus, Sternberg’s conclusion 
must be qualified be stating that his results apply to, for example, 
Lanchester-type processes in which the rate of attrition of each target 
type is proportional to (the sum over all firer classes of) the number 
of firers only (speaking somewhat imprecisely, Lanchester square-law 
processes) . It was not noted by Sternberg that the structure of the 
optimal allocation policy may be different in prescribed duration 
battles and fights to the finish. 

Finally, Bonder and Farrell (see pp. 27-28 of [24]) state that 
(for Lanchester attrition processes in which target type attrition rate 
due to each class of firers is proportional to the number of firers 

* 

In Appendix E of this report we develop a theory of SVIC’s for one- 
sided fire distribution problems in the Lanchester theory of combat. 
Some of the results we give are not to be found in the literature. As 
we have done in the past [115], we hope to publish some of these 
results in the control theory literature. 
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only) the optimal target selection policy is independent of the 
number of weapons in the firer or target group. Our research results 
(see [117] or Section 9 of Appendix G) contradict the above statement 
made by Bonder and Farrell: the optimal fire distribution policy may 

depend indirectly upon force levels. 

Other similar tactical/strategic allocation problems that have 
been studied using differential game models are the supporting weapon 
system game of H. K. Weiss [130], missile warfare (counter-value 
versus counter-force targeting) [105], and the logistics allocation 
game of Moglewer and Payne [95] . We have already noted that Weiss did 
not use the subsequently well-known necessary conditions of optimality 
to develop the solution in his pioneering work [130]. Our previous 
work [122] (see also Appendix B of [113]) corrects this gap. Several 
authors [38], [39], [66] (see also pp. 54-57 of [105]) have considered 
differential equation models of a strategic missile exchange and have 
attempted to study optimal tactics via differential game theory. 
Moglewer and Payne [95] consider a similar model for the optimal allo- 
cation of logistics resources. However, there are many gaps in the 
above work. For example, Moglewer and Payne [95] consider (in our 
terminology presented in [117]) a terminal control differential game. 
However, they fail to determine the domains of controllability for 
terminal states and express optimal tactics in terms of terminal force 
levels . In [117] we showed that it is essential to express optimal 



tactics in terms of initial force levels. 
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It seems appropriate to briefly touch upon the implications 
for operational gaming (see [100] or [124] for discussion of terminology 
and background) of our analytic study of tactical allocation problems. 

We believe that the primary value of operational gaming is in formulat- 
ing the problem. (Accordingly, we are coordinating our analytic efforts 
with the operational study work of P. Chaiken of Stanford Research 
Institute (Naval Warfare Research Center).) Moreover, we agree with 
L. Berkovitz and M. Dresher in their opinion (see p. 612 of [14]) that 
"operational gaming is not a helpful device for solving a game or 
getting significant information about the solution." We believe that 
insights into optimal allocation strategies are to be obtained through 
analytic work such as ours. 

Finally, we could find no previous work on the optimal control 
of the Lanchester stochastic process (i.e. stochastic versions of the 
problems discussed above) . 

4 . Overview of Research Program . 

In this section we briefly discuss the various subject areas 
that we have considered in developing our quantitative theory of 
tactical allocation. 

a. Combat Attrition Models . 

In our research on tactical allocation we have given considera- 
tion to the various different types of models that have been proposed 
(hypothesized) in the literature to describe combat between two homo- 
heneous forces. Such homogeneous force formulations are readily 



extended to describe the 
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attrition for fire distribution problems in combat between heterogeneous 
forces. Our purpose in considering the various modelling alternatives 
for combat attrition has been to determine the sensitivity of optimal 
target selection and force allocation strategies to the form of the 
attrition model. Based upon our research, we conclude that optimal 
allocation policies are significantly affected by the nature of the 
attrition model. 

The reader should recall that there are two general approaches 
which may be taken to the modelling of the attrition process: 

(a) deterministic formulation which takes the form of a system 
of first order ordinary differential equations, and 

(b) stochastic formulation which views the casualty process as 
a Markov process. 

In the remainder of this section we shall discuss deterministic formu- 
lations as related to our study of allocation strategies. Stochastic 
combat attrition models are discussed in the next section. 

Besides our review of the literature above, the interested 
reader can find comprehensive reviews in [41], [46], [56]. The various 
forms that the attrition rates can take in models of combat between 
two homogeneous forces has been discussed by Lanchester [88], Snow 
[109], H. Weiss [129], Brackney [26], Willard [131], and Helmbold [59]. 
(The reader can find a further discussion in Appendix H.) In our past 
research [117], [120], [121] we have studied optimal target selection 
policies when the attrition rate of each target type is proportional 
to: 



34 



(a) the number of firers only, 

(b) the product of the numbers of firers and targets. 

These correspond to the classical Lanchester square-law process and 

linear-law process, respectively. 

R. Helmbold [59] has proposed a general formulation of combat 

attrition for combat between two homogeneous forces (based upon his 

empirical study of a large number of land battles [57]). His general 

model [59] accounts for inefficiencies of scale when force sizes are 

grossly unequal. In this report we present some preliminary results 

for optimal target selection when Helmbold T s general model is used 

to describe attrition in combat between heterogeneous forces. In 

doing this work we have extended his formulation to combat between 

heterogeneous forces and present some analytic solutions. 

b . Deterministic Versus Stochastic Models of Combat Attrition . 

As indicated by our review of the pertinent literature and 

noted above, two basic approaches to analytic war gaming are (1) the 

* 

Lanchester-type deterministic differential equation models [24], [25], 
[50] or (2) stochastic models (such as DYNTACS [16], a high resolution 
simulation) . Almost all previously published research on tactical 
allocation problems (including our own work) has treated attrition 
as a deterministic process. We have briefly discussed work on stochastic 
formulations in the review of pertinent literature above. In Section 2 
* 

These are frequently referred to as expected -value models. 

** 

The one exception of a stochastic optimization problem is the work 
of Isbell and Marlow [73]. 
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of Appendix I the reader can find a more comprehensive review of past 
work on the Lanchester stochastic process. There are, however, con- 
flicting views in the literature as to whether these two approaches 
yield the same results. (We certainly don’t plan to try to settle 
this difficult question, but wish to caution the reader that there 
is not universal agreement among workers in the field about this 
important question.) 

G. Clark has studied (see Chapter 11 of [16] and [41]) the differ- 
ence between deterministic and stochastic homogeneous-force Lanchester 
formulations and concludes that (see p. 11-19 of [16]) "the determin- 
istic model would have difficulty approximating a stochastic simulation" 
with respect to the time history of force levels. Based upon our study 
of Clark’s work [41], we raise the question as to whether there may 
be significant differences between optimal allocation policies for 
deterministic and stochastic attrition processes. If such a difference 
were to be observed, then this would raise the more basic question of 
whether or not different study results could be obtained merely by 
the choice of modelling technique, i.e. deterministic or stochastic 
formulations . 

c . Special Features of Deterministic Optimal Control Models 
for Tactical Allocation . 

In Section 1. above, we have discussed three important problem 
areas in applying deterministic optimal control theory to study tactical 
allocation problems 
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(a) non-uniqueness of extremals, 

(b) state variable inequality constraints, 

(c) singular subarcs. 

(These topics were not discussed in [111].) Most of our research 
effort has been directed at developing a satisfactory treatment of 
these topics for tactical allocation problems. All are relatively 
near the frontiers of research in deterministic optimal control theory. 

Further detailed discussions of the above topics can be found 
in the appropriate appendices which are as follows: 

(a) non-uniqueness of extremals: Appendices A, F, G, 

(b) state variable inequality constraints: Appendix E, 

(c) singular subarcs: Appendix D. 

It seems appropriate, however, to give further general discussion of 
these aspects in order to emphasize their importance in the optimiza- 
tion of combat dynamics via optimal control theory. Additionally, we 
will review their current status within deterministic optimal control 
theory. 

(1) Non-uniqueness of Extremals . 

The purpose of this research is to determine optimal fire dis- 
tribution/target selection policies for tactical allocation problems. 
However, as is well-known, the maximum principle (or the max-min 
principle for two-sided (differential game) problems) provides only 
necessary conditions of optimality. Moreover, let the reader recall 
that we refer to a path on which the necessary conditions of optimality 



37 



(maximum print iple) are almost everywhere satisfied as an extremal . 

Thus, in order to determine the optimal allocation policy one must 
show that a particular extremal is indeed an optimal trajectory 
(otherwise one may incorrectly identify a non-optimal policy as being 
optimal) . 

* 

Two ways of determining optimal trajectories are as follows: 

(a) check whether sufficient conditions of optimality [92], [53] 
are satisfied on the extremal, 

(b) by citing the appropriate existence theorem, show that an 

optimal control exists to the problem at hand; there are two 
further subcases: (1) if the extremal is unique, then it is 

optimal or (2) if the extremal is not unique and only a 
finite number exist, then the optimal trajectory is determined 
by considering a finite number of alternatives. 

It turns out, however, that the former approach is inadequate for most 

tactical allocation problems that we have considered, since existing 

sufficient conditions [92], [53] only apply to optimal control problems 

of fixed length (i.e. fixed terminal time). Without going into subtle 

details at this time, let us state that the results of [92], [53] cannot 

be applied to, for example, the Isbell-Marlow problem [74], [117] 

without qualification (if they are, it is easy to falsely conclude 

that certain non-optimal extremals are optimal) . 

The second approach given above is the one we have taken in this 
research. If an extremal is unique, then it is optimal and no difficulty 



For the purposes of our expository discussion here we are speaking 
somewhat imprecisely in order to communicate to the general reader why 
certain topics have received so much attention in our research. 

This is essentially a condensation of the general solution procedure 
given in Appendix A (and further extended in Appendix G) . 
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exists. However, in the simplest fire distribution problem we have 
shown that extremals may not be unique [117] • (Apparently, this was 
not noted in any past research on this problem [74], [111].) Moreover, 
we have shown how to determine the optimal allocation policy when 
there are multiple extremals in such a problem (see Appendices A and 
F for further details). In the latter case, one may have to contend 
with a dispersal surface (see pp. 132-141 of [72]) being present in 
the solution. 

In our past research, we have shown that dispersal surfaces may 
exist in the solution to even the simplest fire distribution problem 
[117]. This rare singular surface is very difficult to determine, since 
its presence in a solution cannot be determined directly by the maximum 
principle but requires "considerations in the large." Besides the 
examples (all two-sided problems) given in Isaacs’ book (see pp. 135- 
155 of [72]), we are aware of only one other problem [44] studied in 
the literature in whose solution a dispersal surface has been found. 

The existence of multiple (i.e. non-uniqueness of) extremals in 
tactical allocation problems within the Lanchester theory of combat 
has important implications for computational methods (such as dynamic 
programming or other finite difference approximation techniques) . 
Approximate solutions to optimal control problems may be developed 
by discretizing time (i.e. using finite difference approximations) 
and treating the resulting problem as a mathematical programming problem 
[30], [93]. The difficulties caused by the existence of multiple extremals 
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or dispersal surfaces in the solution to a continuous-time optimal 
control problem are not circumvented by considering an approximating 
mathematical programming (finite-dimensional optimization) problem. 

With such subtle features being present in even the simplest 
fire distribution problem, the reader might well ask, !, Can anything 
be done to circumvent such mathematical complexities? 11 We believe 
that this can be partially done by identifying the circumstances under 
which such difficult mathematical points arise in the model's solution. 
In other words, it is very important to have complete (and mathemati- 
cally correct) solutions to the simplest tactical allocation problems 
if for no other reason than to identify which formulations lead to 
prohibitive mathematical complexities (and hence should be avoided) . 

For example, we know that for n-versus-one combat such difficulties 
don't arise when surviving target types are valued in direct proportion 
to their kill capability as measured by their Lanchester attrition- 
rate coefficient. 

Finally, except for the references cited above (and pp. 276-284 
of [4]), we could find no other treatment of multiple extremals in the 
current literature of deterministic optimal control theory. 

(2) State Variable Inequality Constraints . 

As we saw in our discussions in Section 1 above, all tactical 
allocation problems contain negativity restrictions on the force 
levels due to physical reasons (negative force levels don't make sense). 
In general, force levels will be represented by state variables so that 
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such a restriction is mathematically called a state variable inequality 
constraint (frequently denoted as SVIC) . The maximum principle (in 
its original form [101]) is inadequate to handle SVIC's and special 
mathematical theories have been developed for state-variable-inequality- 
constrained problems. When we started our research, we found that 
results were widely scattered in the literature and that a completely 
adequate theory to solve even the simplest fire distribution problem did 
not exist. Hence, we have developed a theory of SVIC ! s for tactical 
allocation problems within the Lanchester theory of combat (see 
Appendix E) . 

A mathematical theory of SVIC ! s is essential for solving tac- 
tical allocation problems within the Lanchester theory of combat. 

Using such a theory, one can determine when it is not optimal to 
drive a force level to zero, the optimal order in which to annihilate 
target types, etc. Thus, the mathematical theory (necessary conditions) 
allows one to (significantly) reduce the volume of allocation strategy 
combinations which one has to examine in solving a differential game 
(thus circumventing the computational difficulty encountered by 
Sternberg (see pp. 154-155 of [111])). 

As noted frequently in this report, no adequate theory of SVIC’s 

* 

exists within the current mathematical theory of differential games. 

* 

A. Friedman, of course, has treated differential games with SVIC s 
(i.e. games with restricted phase coordinates) [52] . His approach is 
different than the variational approach used in optimal control theory, 
and his emphasis is on existence theorems. Moreover, his treatment 
(see pp. 239-240 of [52]) of Isaacs 1 war of attrition and attack (see 
pp. 96-104 of [72]) implicitly requires an unsupported assumption. 
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The development of a mathematical theory of SVIC ! s for differential 
games which occur within the Lanchester theory of combat appears to 
be the most needed extension of the existing theory of differential 
games for military operations research applications. 

Although optimal control problems involving inequality con- 
straints on a function of the state variables with no explicit 
dependence upon the control variables have received fairly compre- 
hensive treatment in recent years (see [77], [79], [94], [115] for 
comprehensive bibliographies) , most results are widely scattered in 
the literature. (Moreover, the theory which has appeared in textbooks 
has some important gaps in it and is inadequate (by itself) to solve 
the fire distribution problems that have been the subject of our 
research [115].) The poineering work of the Russian R. Gamkrelidze in 
1959 (see Chapter VI in [101]) was followed in the USA by that of 
L. Berkovitz [9], S. Dreyfus [47], and Bryson, Denham, and Dreyfus 
[29]. Berkovitz and Dreyfus [15] subsequently have shown the equiva- 
lence of the results of Gamkrelidze and Berkovitz with those of 
Dreyfus . 

McIntyre and Paiewonsky [94] have written an excellent survey 
article on the theory of SVIC's and summarize theoretical results 
through 1966. Subsequent work has been by Jacobson and Lele [77] and 
Jacobson, Lele, and Speyer [79], the latter authors deriving their 
results by considering generalized Kuhn-Tucker conditions in a Banach 
space. Sufficient conditions of optimality for problems with SVIC ! s 
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have been given by Funk and Gilbert [53]. Finally, we would like 
to mention the work by two students of M. Hestenes [55], [102]. 

Moreover, as we have recently pointed out [115], practitioners 
of optimal control have not always been aware of the precise content 
of the above work. We are aware of no previous work relating the 
theory of SVIC's to allocation problems in the Lanchester theory of 
combat . 

(3) Singular Subarcs . 

As we have seen in Section 1 above, tactical allocation problems 
within the Lanchester theory of combat may be formulated (see equations 
(1) above) as differential games in which the strategy variables (e.g. 
fraction of who fire at Y^) appear linearly . Such problems 

are called singular (because the matrix of second partial derivatives 
of the Hamiltonian with respect to the strategy variables is a singular 
matrix) and require special treatment. One-sided versions of such 
fire distribution problems then are singluar problems of optimal 
control. A discussion of terminology and the mathematical aspects of 
singular optimal control problems has been given in Section l.b. above. 
We saw there that the maximum principle does not determine the singular 
control and that the classical theory does not provide adequate tests 
of optimality for singular subarcs. 

A mathematical theory of singular subarcs is essential for 
solving tactical allocation problems within the Lanchester theory of 
combat because it is required to determine the optimality of fire 
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distribution policies that are not extreme points of the control 
variable space (i.e. policies that are other than 0 or 1). We 



have seen that an optimal fire distribution policy, denoted as <j> , 

k 

such that 0 < <j> <1 arises in the simple problem of target selec- 

tion for a homogeneous force in Lanchester combat against two enemy 
force types each of which undergoes attrition at a rate proportional 
to the product of the numbers of firers and targets [120] . 

The establishment of the singular subarcs in such a problem as 
above requires examination of the generalized Legendre-Clebsch condi- 
tion (a second order condition) . In a problem such as (11) in which 
target-type attrition corresponds to Lanchester f s classical square-law 
attrition process (i.e, the attrition rate of a target type is propor- 
tional only to the number of firers) the absence of singular subarcs 
is established by showing that it is impossible to have (except for 



The importance of the generalized Legendre-Clebsch condition is empha- 
sized by the fact that we have encountered a problem (suggested by 
the work of P. Chaiken) in which it is the only way to establish the 
non-optimality of splitting forces in combat. 



explicitly studied in 1963 by Johnson and Gibson [81] (see [80] for 
a discussion of singular solutions in optimal control problems within 
a broad perspective of applied mathematics) . Since that time the 
subject has grown to comparative maturity although most recent results 



pathological cases) 




where H denotes the Hamiltonian. 



Singular problems of optimal control were apparently first 
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are widely scattered in the literature. Surveys of early work are 
contained in the works by Kelley, Kopp, and Moyer [83] (summarizes 
results through 1966) and Bryson and Ho (see Chapter 8 of [28]) 
(summarizes results through 1968) . The above work is sufficient to 
adequately treat all the singular problems of the optimal control of 
deterministic Lanchester processes which we have so far encountered. 

Kelly [82], [83] apparently first generalized the Legendre- 
Clebsch necessary conditions to singular subarcs. This was further 
generalized to singular extremals involving several control variables 
by Goh [54]. Jacobson later derived an additional necessary condition 
via differential dynamic programming [75]. Very recently, Jacobson 
[76] has given a general sufficiency theorem for the second variation 
which allows one to treat both singular and non-singular problems. 

This yields conditions analogous to the well-known no-conjugate- 
point condition (see pp. 181-184 of [28]) for singular subarcs and has 
led to necessary and sufficient conditions of optimality for singular 
control problems [110] (see also [78]). 

Except for our own work [121] , we are aware of no previous 
work applying the theory of singular extremals to tactical allocation 
problems within the Lanchester theory of combat. 

d . Optimal Control of the Lanchester Stochastic Process . 

A totally underdeveloped area in the Lanchester theory of combat 
has been optimal target section in combat among heterogeneous forces 
when attrition is a stochastic process. Based on our review of the 
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pertinent literature (see Section 1 of Appendix I) , we conclude that 
there has been no past adequate treatment of stochastic versions of 
the fire distribution problems that we have previously studied 
using deterministic models. By studying such stochastic versions, 
we plan to see if the "best" tactics for target selection are depend- 
ent upon how attrition is modelled, i.e. deterministic or stochastic 
process. In this report we present preliminary results (development 
of the fundamental functional equation for the optimal expected value 
function, its analytic solution in some special cases, and a numerical 
approximation for the general case) . Although our results are special 
cases of well-known results in the field of optimal stochastic control, 
the optimal control of the Lanchester stochastic process had apparently 
never been studied in the past by this approach. 

In Appendix I we consider a battle which is to last a prescribed 
length of time as an optimal stochastic control problem. Our idea is 
to compare the structure of the optimal allocation policies between 
deterministic and stochastic formulations. Accordingly, we have 
developed a complete "solution" to the corresponding deterministic 
control problem (see Appendix G) . We are currently directing a 
graduate student (LT R. Powers, USN) in making a numerical comparison 
between results obtained by these two models, and we will report 



these results at a later date. 
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5 . Guided Tour of the Appendices . 

In this section we summarize the work which is contained in 
the appendices and explain why this work was done. In our first report 
[113] we emphasized the extension of results which had previously 
appeared in the literature. In the present report we give further 
extensions and, moreover, develop the mathematical foundations for 
more extensive future work. 

In Appendix A we derive a complete solution to the Isbell and 
Marlow fire programming problem [74]. This current work revises 
earlier preliminary results on this problem. We develop general 
methodology for solving such deterministic optimal control problems 
and then demonstrate its application by developing a complete solution 
(however, further extensions are given in Appendix F) for a terminal 
control battle (the battle only terminates when the course of battle 
has reached some specified state) between a homogeneous force and a 
heterogeneous enemy force of two target types. Our methodology is 
more general, though, and may be applied to the cases of one-versus-n 
target types, variable attrition-rate coefficients, etc. 

The structure of the optimal fire distribution policy for the 
Isbell-Marlow problem is discussed. We show how the optimal fire 
distribution policy may depend indirectly upon the force levels. For 
the attrition model used (attrition rate of target types proportional 
to the number of firers only) the optimal policy is always to concen- 
trate all fire on one target type. Our work extends that of Isbell 
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and Marlow [74] by showing that the solution to this optimal control 
problem contains dispersal furfaces . Additionally, we show how to 
determine the domain of controllability for extremals to each terminal 
state. 

The work presented in Appendix A was done because the Isbell- 
Marlow problem [74] represents the simplest terminal control fire 
distribution problem. Any mathematical theory developed to handle 
more complex tactical allocation problems of greater practical signi- 
ficance must be able to adequately handle this simple case. 

In Appendix B we examine the structure of optimal allocation 
policies for tactical situations described by Lanchester-type models 
of warfare by studying a sequence of simplified models. We do this 
because no previous work had ever systematically investigated the 
dependence of tactics upon model form. We consider models for two 
types of choice situations 

(1) selection of target type, 

(2) regulation of firing rate. 

These problems are solved by the mathematical theory of optimal 
control . 

In the first sequence of models we examine the effects on the 
optimal target selection policy of the following factors: objectives 

of the combatants, termination conditions of the conflict, number of 
target types, some special cases of time-dependent attrition-rate 
coefficients, and type of attrition process. We then examine a sequence 
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of models to see how ammunition limitations affect firing rates. Next 
we discuss two-sided extensions of such problems but point out the 
value of studying one-sided problems as considered in this report. 
Finally, various implications of the models are discussed. Some of 
the results presented in this appendix are further extended in 
Appendix G. 

In Appendix C we develop partial solutions to a sequence of 
models for the optimal fire distribution policy for a homogeneous 
force in combat against geterogeneous enemy forces. We consider a 
prescribed duration battle and concentrate on the special case when 
no force type is annihilated during combat. For both one-versus-two- 
target-types combat and one-versus-n-target-types combat we develop 
our results for both constant attrition-rate coefficients and also 
some special cases of variable attrition-coefficients. 

We undertook the work of Appendix C to show how our results 
for one-versus-two-target-types combat could be extended to one- 
versus-n combat and variable attrition-rate coefficients (thus closer 
modelling real world complexities). In this work we draw heavily 
upon our research on homogeneous force models with variable 
attrition-rate coefficients [114], [116], [123] (see also Appendix 
H) . First, we consider the problem of optimal fire distribution over 
two target types for some special cases of variable attrition-rate 
coefficients. Next, we consider several target types and constant 
attrition-rate coefficients. Then, we treat the more general case 
of several target types for some special instances of variable 
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attrition-rate coefficients. Finally, we make some observations on 
the results presented in this appendix. 

Our results presented in Appendix C show that some special 
cases of variable attrition-rate coefficients are mathematically 
tractable for optimal control problems within the Lanchester theory 
of combat. Additionally, we show that in one-versus-n combat the 
solution to the optimal fire distribution problem takes a particularly 
simple form when enemy target types are valued (linearly) in direct 
proportion to their kill capability (as measured by the Lanchester 
attrition-rate coefficient) . In this special case, the optimal fire 
distribution policy is particularly simple: namely, concentrate all 

fire upon the available target type with the largest product of 
attrition-rate coefficients (i.e. largest a_jb ) . This is even true 
for some special cases of variable attrition-rate coefficients. 

In Appendix D we treat a simple problem of fire distribution 
for a homogeneous force in Lanchester combat against two enemy force 
types, each of which undergoes attrition at a rate proportional to the 
product of the numbers of firers and targets. We undertook this work 
in order to study the dependence of optimal tactics upon the attri- 
tion model. In addition to the Pontryagin maximum principle, the 
theory of singular extremals is required to solve this problem. 

Our major result is that for fire distribution problems in 
which target-type attrition corresponds to the classical Lanchester 
linear-law attrition model "the optimal fire distribution policy is 
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not always to concentrate all fire on a single target type . There are 
circumstances when the optimal policy is to split one's fire between 
the two target types. Additionally, we show how to synthesize the 
optimal fire distribution policy from the basic optimality conditions. 
For constant attrition-rate coefficients we show that whether or not 
changes can occur in target priorities depends solely on how survivors 
are valued and is independent of the type of attrition process. 

In Appendix E we summarize results for the theory of state 
variable inequality constraints (frequently denoted as SVIC*s) which 
had previously been widely scattered in the literature. Then we apply 
this theory to two tactical allocation problems. The first problem 
(tactical air-war campaign) is a one-sided version of R. Isaacs 1 
"War of Attrition and Attack” [72]. However, the non-negativity of 
the force levels was not adequately handled mathematically in past 
work [72], and we consider versions of the problem for which the 
previous method of analysis does not lead to an optimal policy. The 
second problem is the fire programming problem studied by Isbell and 
Marlow [74]. We consider several versions of both these problems. 

For one-versus-n-target-types combat we extend our past results by 
developing necessary conditions of optimality for annihilating a 
target type in two important cases: (1) two target types with a 

special case of variable attrition-rate coefficients and (2) n 



target types with constant attrition-rate coefficients. 
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We undertook the work presented in Appendix E since no past 
work on optimal tactical allocation had given an adequate treatment 
of the non-negativity restrictions on force levels in developing 
optimal policies. In our own previous work on the Isbell-Marlow 
problem [74] we followed the heuristic treatment originated by Isbell 
and Marlow [74] (who, in turn, were following Isaacs [70]) for deter- 
mining the boundary conditions for the dual variables. Our work here 
establishes a firm theoretical foundation for such important aspects 
and justifies certain assumptions that we made in developing our 
solution to the Isbell-Marlow problem [117]. Thus, we re-consider 
the development of certain key necessary conditions of optimality 
within the framework of the theory of SVIC T s. 

In Appendix F we consider some further aspects of the solution 
to the Isbell and Marlow fire programming problem. We show how the 
Isbell-Marlow problem is solved using the theory of SVIC T s. Addition- 
ally, we show that for certain values of model parameters (1) the 
optimal policy may not be unique and (2) in treating multiple extremals 
there is a dominated return (i.e. the return associated with one family 
of extremals dominates that associated with another family of extremals) 
besides the dispersal surfaces in this problem 7 s solution. We under- 
took this work because our treatment of more complex tactical allocation 
problems relies heavily on the complete solution of simpler problems. 
This important theory (SVIC f s)had never been applied previously to 
determine the optimal control of deterministic Lanchester processes. 
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In Appendix G we develop a complete solution to the prescribed 
duration battle using the theory of SVIC’s. We do this in order to 
compare the structure of the optimal fire distribution policy for the 
prescribed duration battle problem with that of other problems (i.e. 
terminal control battle and optimal control of the Lanchester stochas- 
tic process) . We give a general solution algorithm that applies to 
all such deterministic optimal control problems. We have used this 
algorithm to generate all the results given in this report. Discus- 
sions of the following important topics are given : (1) the structure 

of the optimal fire distribution policy and (2) the development of 
solutions to such problems. We show that even when target types 
undergo a "square-law" attrition process, the optimal allocation 
policy may depend indirectly upon the force levels . 

In Appendix H we develop solutions to extensions of F. W. 
Lanchester’s classical equations of modern warfare (frequently referred 
to as aimed-fire equations) for combat between two homogeneous forces. 
In these extensions the lethality of the fire (as expressed by the 
Lanchester attrition-rate coefficient) depends upon time. When the 
dependence is arbitrary, the solution is an infinite series of recur- 
sively related integrals; in special cases, more convenient representa- 
tions (including representation in terms of tabulated functions) are 
available. Solutions are obtained in the following cases: (1) 

lethality of each side’s fire proportional to a power of time and both 
lethalities initially zero, and (2) lethality of each side’s fire 
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linear with time but only one side's lethality initially zero. The 
latter case models the constant speed approach between forces whose 
weapons have different maximum effective ranges. We undertook this 
work because we could find no adequate treatment available in the 
literature. These results have been subsequently used for problems 
on the optimal control of deterministic Lanchester processes with 
variable attrition-rate coefficients . 

In Appendix I we present preliminary results on the optimal 
control of the Lanchester stochastic process. We consider a stochastic 
version of the prescribed duration fire distribution problem studied 
in Appendix G. Our purpose for doing this work is to determine 
whether the structure of the optimal allocation policy is affected by 
whether the attrition process is modelled as being deterministic or 
stochastic . 

Thus, in Appendix I we develop the fundamental functional 
equation satisfied by the optimal expected value function and derive 
solutions in some special cases. Except for these special cases, 
the fundamental functional equation (actually a system of differential- 
difference equations) possesses a solution that is too complex for the 
development of a closed-form analytic solution. Therefore we have 
developed a finite difference approximation which can be used to 
generate numerical solutions for small numbers of combatants. 

Using the above finite difference approximation, a thesis student 
has done some numerical computations for us. It has only proven to 
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be feasible to use the computer to generate numerical results for 
battles with up to 10 Y's versus 10 1 s and 10 1 s due to 

computer memory requirements. Additionally, we have considered 
approximations valid for large numbers of combatants. In pursuing 
this approach, we have developed a diffusion approximation to the 
Lanchester stochastic process. 

In the fire distribution problems studied above for a homog- 
enous force in combat with a heterogeneous enemy it has always been 
assumed that fire can always be instantaneously re-distributed from 
being entirely concentrated on one target type to any other one. 

For example, in M two-on-one n combat the decision (or control) variable 
represents the fraction of Y fire that is directed against X^ . 
Instantaneous jumps in this control variable (for example, from 0 
to 1) have been permitted. This corresponds, in a sense, to a 
"perfect” command and control capability. In Appendix J we present 
preliminary results for the more realistic assumption that there is 
a limit to how fast <j) can be changed corresponding to command and 
control limitations . 

We undertook this work presented in Appendix J in order to 
explore the effects of command and control capabilities upon the 
structure of the optimal fire distribution policies. We show that 
the presence of command and control limitations (which cause the rate 
of re-distribution of fire to be bounded) causes fire to be shifted 



earlier than when fires can be instantaneously re-distributed in 
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anticipation of changes in target priorities. This phenomenon is 
analogous to the situation when with reduced reaction times, one 
tries to "anticipate" the actions of an enemy. The development of 
these results has required some advanced theory for SVIC T s (treatment 
of a second order SVIC) . 

Finally, in Appendix K we further investigate the sensitivity 
of the optimal target selection policy to the form of the combat 
attrition model by considering a two-against-one-target-type selec- 
tion problem with combat attrition following a general form proposed 
by R. Helmbold. For combat between homogeneous forces Helmbold [59] 
has proposed a general Lanchester-type model in which the effective- 
ness of a force is dependent upon the force ratio. This general 
formulation yields Lanchester*s square law and Lanchester*s linear 
law as special cases. We extend Helmbold* s model to combat between 
heterogeneous forces and use this extension as the attrition model 
in a tactical allocation optimization problem. 

In Appendix K, then, we present preliminary results for optimal 
target selection for Helmbold *s general attrition structure. These 
preliminary results indicate that when target-type attrition follows 
Helmbold T s general model, the optimal policy is always to concentrate 
all fire on a single target type . (In other words, we could find no 
singular solution in this case.) This should be contrasted with our 
result [121] that when target types undergo attrition corresponding to 
a "linear-law" Lanchester process, it is sometimes optimal to split 
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one f s fire. In other words, as far as the optimal distribution of fire 
is concerned, Helmbold f s general attrition structure behaves like 
that in which a target type's attrition rate is proportional only to 
the number of firers. 

6 . Relevance to Current Navy Problems . 

In this section we explain the relevance of our research on 
the optimal control of Lanchester processes to current Navy problems. 

a. Our theoretical work on tactical allocation problems is 
being interfaced with the ONR supported study effort of P. Chaiken of 
Stanford Research Institute. Chaiken has pioneered [31], [32], [33], 
[34] in the application of quantitative (Lanchester-type) methodology 
for force level planning, development of contingency plans, etc. This 
methodology has been used for significant force level problems defined 
by the military — Commander-In-Chief-Pacific (CINPAC) , Headquarters 
Marine Corps (HQMC) , and Chief of Naval Operations (CNO) [34]. 

To determine optimal allocation strategies in dynamic problems, 
Chaiken uses a heuristic optimization approach of assuming a certain 
type of strategy (constant or piecewise constant) and then comparing 
computed campaign outcomes for various levels of strategy variables 
(essentially, a so-called direct method of optimization [90]). Our 
research supports his study objectives by (1) justifying such assump- 
tions, (2) suggesting strategy alternatives to explore in more detailed 
computer simulations such as BALFRAM [37], (3) explicitly showing the 
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effects of modelling assumptions on optimal tactical allocation 
strategies, (4) developing insights into optimal allocation strate- 
gies for various attrition structures of interest. 

b. Thus, we are striving to provide a sound theoretical basis 
for application efforts such as by Chaiken. In our present work we 
have concentrated on solution techniques and methodology, since this 
was a gap in the current state-of-the-art for applications of optimal 
control/differential game theory to Lanchester-type models of warfare. 
We have studied idealizations (abstractions) of allocation structures 
arising in more complex (realistic) defense planning studies. Through 
such idealization and/or simplification we have tried to identify 
factors or concepts which merit further investigation in more detailed 
computer simulations (such as BALFRAM [37]). For example, uses have 
never exercised the "linear-law" attrition option of MILISTRAC and 
BALFRAM [36]. Yet our research indicates that optimal tactics with 
"linear-law" attrition are fundamentally different than those with 
’’square-law" attrition. Hence, planners should be alerted (educated) 
to this fact about the dependence of optimal tactics on type of 
attrition process. 

c. It seems appropriate to briefly discuss a modelling issue 
of abstraction versus realism. The mass of details present in a 
realistic (complex) model inevitably obscures essential modelling 
issues such as type of attrition process, type of objective (criterion) 
function. Our work is oriented towards clarifying basic issues by 
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examining simplified models. For example, versions of a well-known 
model (see pp. 96-104 in [72]) which are used for justifying tactics 
in the air war consider a combat payoff in terms of net sorties of 
ground support missions flown and totally ignore the effects of such 
sorties on the outcome of the ground war. Our theoretical analysis 
of this situation provides motivation (and justification) for a 
current ONR study [35] to determine the optimal tactics of the air 
war in the context of the ground war objectives for contingency plans 
of interest. 

d. A sequential game arises in a current application of P. 
Chaiken ! s BALFRAM methodology [35]. Our research lays the theoretical 
foundations (via the research on differential games) for the inclusion 
of the capability to handle such a sequential game within the BALFRAM 
software system. 

e. The questions of target priority rules for target selection 
frequently arises in the evaluation of proposed weapon systems and/or 
doctrine of employment for weapon systems. The study of idealized 
combat models would provide insight into the optimization of combat 
dynamics hopefully leading to a better quantitative basis for defense 
planning decisions. 

f. In the Navy mission of fire support, the question of target 
priority rules for target selection frequently arises. An understanding 
of optimization principles in various dynamic combat situations would 



lead to better utilization of such resources. 
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7 . Summary of Research Findings . 

Here we summarize our research results. We found that solution 
techniques were insufficiently developed for the optimization of 
combat dynamics. Accordingly, this aspect received top priority in 
our work. Results are organized under the following headings: 

(1) solution techniques, 

(2) insights into optimal tactical allocation, 

(3) implications for defense planning. 

Items (2) and (3) differ in that the latter is a management-oriented 
digest of the practical implications of our research whereas the 
former is oriented towards a technical audience. Further amplification 
of results and conclusions is to be found in the appendices. 

a . Solution Techniques . 

Our research has produced the following results on solution 
techniques for the optimization of combat dynamics. Specifically, we 
have accomplished the following: 

(1) developed general solution algorithm (based on results from 
modern optimal control theory) to determine the optimal 
control of deterministic Lanchester processes, 

(2) developed the basic equations of optimality for the optimal 
control of the Lanchester stochastic process, 

(3) developed a theory (theory of state variable inequality 
constraints) for the adequate treatment of non-negativity 
restrictions on force levels in optimal control problems 
for deterministic Lanchester processes, 

(4) discovered that no adequate theory of state variable inequality 
contraints exists in current theories for differential games, 
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(5) shown how to treat the following difficult aspects of optimal 
control problems for deterministic Lanchester processes: 

(a) singular solutions, 

(b) state variable inequality constraints, 

(c) multiple extremals (including dispersal surfaces and 
dominated returns) , 

k 

(6) developed complete solutions for optimal fire distribution 
tactics in one-against-two-target-types Lanchester combat for 
two cases: 

(a) terminal control battle (f ight-to-the-f inish) , 

(b) prescribed duration battle, 

■k 

(7) developed partial solutions for optimal fire distribution tactics 
for a force engaged in Lanchester combat for the following 
cases : 

(a) one-against-two-target-types combat , 

(A) some special cases of variable attrition-rate 
coefficients , 

(B) u linear-law !! attrition of target types, 

(C) bounded rates for changing the distribution of fire, 

(D) Helmbold T s general attrition model, 

(E) stochastic attrition process, 

(b) one-against-n-target-types combat, 

(A) constant attrition-rate coefficients, 

(B) some special cases of variable attrition-rate 
coefficients , 

(8) developed solutions to homogeneous force models with variable 
attrition-rate coefficients. 



In (6) and (7) the attrition process is deterministic and the attrition 
rate of a target type is proportional to only the number of firers 
unless otherwise noted. 
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b. Insights into Optimal Tactical Allocation . 

Based on our study of the optimization of combat dynamics using 
the mathematical theories of differential games/optimal control we 
have reached the following conclusions: 

(1) the structure of the optimal allocation policy for fire 
distribution/ target selection depends upon model form ; more 
specifically, the optimal policy depends on force levels, 
weapon system capabilities (as measured by Lanchester attri- 
tion-rate coefficients), the type of attrition process, the 
target acquisition process, the values placed upon surviving 
force types, and the termination conditions of combat, 

(2) for deterministic Lanchester one-versus-n-target-types 
combat in which the attrition rate for each enemy target type 
is proportional to only the number of firers, the optimal 
fire distribution policy has the following properties (for 
linear utility of survivors and no replacements) : 

(a) fire is always concentrated on a single target type 
(which may change without the annihilation of the 
previous target type) , 

(b) it may depend indirectly upon force levels (depending 
upon how survivors are valued) , 

(c) when enemy survivors are valued in direct proportion to 
their kill capability (as measured by their Lanchester 
attrition-rate coefficient against the homogeneous force), 

(A) for constant attrition-rate coefficients 

(I) fire is always concentrated on the available 
enemy target type which has the largest product 
of attrition-rate coefficients (i.e. the largest 
a.b. where a. represents the kill capability 
o£ the friendly force against the enemy f s ith 
target type and b. represents that of the 
enemy’s i th force type), 

(II) fire is only shifted to a new target type after 
the annihilation of the previously available 
target type with the largest product of attri- 
rate coefficients (i.e. no change in fire dis- 
tribution without the annihilation of a target 
type) , 
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(B) for variable attrition-rate coefficients in the 

case when the ratio of enemy weapon system capabili- 
ties (as measured by Lanchester attrition-rate 
coefficients) is a constant for any two enemy 
weapon system types 

(I) assuming that no target type is annihilated, 
all fire is always concentrated on the target 
type with the largest product of attrition- 
rate coefficients, 

(II) additional assumptions are required to analyze 
the case when a target type is annihilated, 

(d) target priorities are not directly influenced by replace- 
ment rates when these are added to the model, 

(3) for deterministic Lanchester one-versus-two-target-types 

combat in which the attrition rate for each enemy target type 
is proportional to the product of the numbers of firers and 
target types, the optimal fire distribution policy has the 
following properties (for constant attrition-rate coefficients, 
linear utility of survivors, and no replacements): 



(a) it depends directly upon force levels, 

>v 

(b) <j> (the optimal fraction of Y-fire directed at X ) 

takes on one of three values: 0,1, or sl ^/ \ 

thus, it is sometimes optimal to split one's fire, 

(c) when enemy survivors are valued in direct proportion to 
their kill capability (as measured by their attrition- 
rate coefficients against the homogeneous force) 



(A) the ranking of target priorities is never reversed 
over time, 

(B) the optimal policy is given explicitly by 
1 



a l +a 2 



for 


a,b,x. 


> 


a„b^x„ , 


Ill 




2 2 2 


for 


a, b, x. 


= 


a„b„x„ , 
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2 2 2 


for 


a, b, x. 


< 


a„b.x„ , 
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2 2 2’ 



0 



(d) the structure of the optimal policy is not sensitive to 

the type of attrition sustained by the homogeneous force, 
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(4) for deterministic Lanchester one-versus-two-target-types 
combat in which the attrition rate for each target type is 
proportional to only the number of firers with (a) constant 
attrition-rate coefficients, (b) linear utility of survivors, 
and (c) no replacements, the optimal fire distribution policy 
may be different in the following two cases: 

(a) prescribed duration battle, 

(b) f ight-to-the-f inish , 

(This difference can only occur when for a^b^ > a 2^2 eneray t ar g e t 
type two is valued in greater proportion to its kill capability than 
target type one is.) 

(5) for the stochastic version of the prescribed duration battle 
described in (4) above, the optimal fire distribution policy 
sometimes depends upon the force levels, 

(6) for the battle described in (4) above, when there is an upper 
bound on the rate of changing one’s distribution of fire 
(through command and control limitations) , the structure of 
the optimal fire distribution policy is not appreciably 
changed;, the following statements can be made about the 
optimal policy: 

(a) the same conditions remain for it to be optimal to have 
<j>* = 0 or 1 (although clearly <j>* may have to take 
on intermediate values) , 

(b) one starts to shift fire earlier (as measured by the 
rate of destruction of target type value, i.e. product 
of attrition-rate coefficient and marginal value of 
target type) than in the previous case when <f>* can 
instantaneously jump, for example, from 0 to 1, 

(7) the optimal fire distribution policy does not appear to be the 
same for models with a deterministic attrition process and 
those with a stochastic attrition process. 

c . Implications for Defense Planning . 

In our research reported here we have studied abstractions 
(idealizations) of allocation structures that commonly occur in defense 
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planning studies. After studying these simplifications in order to 
obtain significant information about the structure of the optimal 
allocation strategies, we have reached the following conclusions 
(generalizations of phenomena that we have observed in simplified 
cases) as to considerations that should be brought to the attention 
of defense planners. These results should be kept in mind by 
practitioners who perform more detailed computer simulation studies. 

(1) The nature of the attrition process has a significant effect 
upon optimal strategies. At present we feel that the highest 
priority should be given to educate planners to the fact that 
the type of optimal allocation strategy is dependent on the 
nature of the attrition model. This has the operational 
planning implication that studies should be run with both 
square-law attrition and then linear-law attrition. Past 
history shows that, for example, the linear-law attrition 
option has never been exercised in BALFRAM [36]. 

(2) Force levels do affect optimal strategies. Whether one "wins” 
(superiority) or "loses” (inferiority) affects optimal 
strategies . 

(3) Even the nature of the scenario (terminal control n pre- 
scribed duration conflict) may affect optimal strategies. 

Thus, if one develops "good" tactics for a 90 day European 
campaign, such tactics need not be "good" if the conflict 
does not terminate at the prescribed time. 

(4) Optimal tactics for the air war are dependent upon whether or 
not the air war is evaluated in the context of ground war 
objectives . 

(5) Optimal tactics are also significantly influenced by the 
nature of the target acquisition process and command and 
control capabilities. 
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8 . Suggested Future Research Tasks , 

After performing the research documented in this report, we 
feel that the current state-of-the-art for applying optimal control/ 
differential game theory to determine optimal allocation strategies 
for deterministic Lanchester processes is such that more significant 
results may be readily obtained in the future. We do not feel that 
previous applied research has been on a sound theoretical basis, and 
we hope that our work here has established that basis. With the 
development of a theory of state variable inequality constraints 
(SVIC T s) for differential games we feel that the capabilities for 
solving applied problems would be appreciably increased. Therefore, 
we give this task top priority. 

Based on our past research experience we feel that there is 
much to be accomplished in the future. Specifically, we suggest the 
following as future research tasks: 

(a) Development of a theory of state variable inequality constraints 
for Lanchester-type differential games. 

(b) Development of further results for deterministic Lanchester 
one-versus-n-target-types combat. This work would build upon 
past research results and would include 

(1) development of more complete solutions (for cases of both 
constant attrition-rate coefficients and also variable 
attrition-rate coefficients) , 

(2) incorporation into the models of such additional factors 
as replacements, logistics, and supporting weapons, 

(3) further consideration of "linear-law" attrition of 
target types. 
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(c) Study of the optimal division of forces to engage target 
types. A slightly different attrition model than in the 
Isbell-Marlow fire distribution problem is one in which the 
friendly forces (assumed homogeneous) only suffer casualties 
when they engage an enemy target type. (In the Isbell-Marlow 
problem the friendly forces sustain casualties from an enemy 
target type regardless of whether or not they engage it.) 

This model is especially important, since it can be used to 
answer several questions that have arisen in the work of P. 
Chaiken . 

(d) Development of simple (probably non-optimal) fire distribution 
policies as approximations to the optimal policy. The worth 
of such an approximate optimal policy would be evaluated by 
comparing the payoff associated with it to the optimal return. 
Besides the intrinsic importance of this aspect, we suggest 
this task, since we have developed complete solutions to 
several deterministic Lanchester one-versus-two-target-type 
combat problems. 

(e) Examination of other one-sided allocation problems of tactical 
interest in the Lanchester theory of combat (for example, the 
optimal distribution of supporting fires (such as Naval fire 
support) over enemy units in Lanchester combat with friendly 
forces) . Emphasis would be on solution development for each 
of various modelling alternatives in order to study the 
dependence of optimal tactics on model form. 

(f) Performance of computational studies on the optimal control 
of the Lanchester stochastic process. Using the finite 
difference approximation developed in Appendix I, we would 
compare optimal strategies for deterministic and stochastic 
Lanchester processes. Furthermore, we would develop the 
fundamental functional equation for the terminal control 
battle (in Appendix I we consider the prescribed duration 
battle) and again make such a comparison. Theoretical inves- 
tigations should concentrate on gaining more insight into the 
structure of the optimal target selection policy when the 
attrition mechanism is stochastic. 

(g) Further study of the optimal control of deterministic Lan- 
chester processes when there is a bound on the rate of change 
of the distribution of fire. Results of our initial effort 
on this topic are contained in Appendix J. We suggest exten- 
sion of these preliminary results. 
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(h) Further study of the dependence of the optimal fire distribution 
policy on the nature of the attrition model. We have given 
some preliminary results in Appendix K for Helmbold's general 
attrition structure. We suggest further extension of this 
work and consideration of still other attrition models (for 
example, that of Willard [131]). 

(i) Examination of optimal tactics for other objective functions 
(i.e. performance (effectiveness) criteria). For all the work 
contained in this report the objective function has been a 
linear function of the number of survivors. Other objective 
functions that might be considered are 

(1) nonlinear valuation of survivors (such as quadratic as 
motivated by Lanchester's square law), 

(2) the value of losses rather than survivors. 

(j) Consideration of some simple Lanchester-type differential 
games. All previous work has been handicapped by the lack 

of an adequate theory of SVIC's for differential games. Such 
a theory can be used to reduce the volume of allocation 
strategy combinations that must be considered. Previous 
work in the literature for which the non-negativity restric- 
tions on the force levels has not received adequate treatment 
is as follows: 

(1) the war of attrition and attack of R. Isaacs [72], 

(2) the supporting weapon system game of H. K. Weiss [130], 

(3) the logistics allocation game of Moglewer and Payne [95], 

(4) the "two-on-two" differential game studied by S. Sternberg 

[ 111 ]. 

After the development of a theory of SVIC ! s for Lanchester 
differential games, it would seem appropriate to re-examine 
the above work. Results from studying these problems are 
useful to the work of P. Chaiken. 

(k) Study of the optimal tactics for the air war in the case when 
the air war is evaluated in the context of ground war objectives. 

Further details on proposed future work is to be found in the 

discussion sections of Appendices E, G, I, J, and K. 
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Appendix A. On the Isbell and Marlow Fire Programming Problem. 



1 . Introduction . 

In this appendix a complete solution is derived to the Isbell 
and Marlow fire programming problem. The original work of Isbell 
and Marlow has been extended by determining the regions of the 
initial state space from which optimal paths lead to each of the 
terminal states of combat. The solution process has involved 
determining the domain of controllability for each of the terminal 
states of combat and the determination of dispersal surfaces. This 
solution process suggests a solution procedure applicable to a wider 
class of tactical allocation problems, terminal control attrition 
differential games. The structure of optimal target engagement 
policies in "fights to the finish" is discussed. 

An underdeveloped area [6] of the Lanchester theory of combat 
is target selection for combat among heterogeneous forces. This type 
of problem has been studied by Isbell and Marlow, who considered both 
a truncated stochastic (Lanchester) process by game theoretic means 
[12] and a terminal control (one-sided) differential game [11]. An 
attrition differential game is an idealized combat situation described 
by Lanchester-type equations [5], [18] over a period of time with 
choices of tactics available to both sides and subject to change with 
time. Terminal control attrition games only end when the course of 
combat has been steered to a prescribed state. 
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In developing a theory of target selection it is important 
to understand the dependence of allocation rules on the type of model 
chosen. Tactical allocation problems may be studied in two types of 
scenarios: (1) the prescribed duration battle and (2) the terminal 

control battle (a particular case of which is the !I fight to the 
finish 11 ) • All the attrition examples in Isaacs 1 book [14] are of the 
first type (his "War of Attrition and Attack" is the continuous 
version of the tactical air war game [3], [4], [9] studied at RAND). 

Only Isbell and Marlow [11] and Weiss [19] have studied the terminal 
control problem. Unfortunately, Isbell and Marlow did not obtain 
a complete solution to their problem. They could not determine when 
certain terminal states of combat were reached. Weiss studied a problem 
which may be considered to be a generalization (two-sided version) of 
their problem. Writing many years before results were known beyond 
a small number of researchers, he did not use the subsequently well- 
known necessary conditions [1]. That his solution procedure [19] was 
heuristic is not surprising, since the simpler problem* [11] which 
he referenced in his paper had not been completely solved. 

ft 

Since the original version of this paper, this problem has also been 
recently studied by S. Sternberg [17]. He employed a more geometric 
approach closely following that of R. Isaacs [14] and originally 
employed by Isbell and Marlow [11]. Sternberg obtained different 
results than those presented here. 
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Thus the obtaining of a complete solution to the Isbell and 
Marlow problem is viewed as a first step leading to the solution of 
more general tactical allocation problems. In this present appendix, we 
shall derive its solution by the Pontryagin maximum principle [16]. 
Generalizations of this procedure to differential games are indicated, 
and solution properties are discussed. In view of the close connec- 
tion [1], [10] between optimal control and differential games (Isaacs), 
the terminology of these two fields is used somewhat interchangeably. 

2 . The Fire Distribution Problem . 

The situation considered by Isbell and Marlow [11] is the 
simplest problem of fire distribution: combat betwee an X-force at 

two force types (for example, riflemen and grenadiers) and a homo- 
geneous Y-force (for example, riflemen only) . This situation is 
shown diagrammatically below. 




1 - cf> 
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It is the objective of the Y-force commander to maximize his survi- 
vors at the end of battle and minimize those of his opponent (consid- 
ering the utilities assigned survivors) . This is accomplished through 
his choice of the fraction for fire, <p , directed at . The battle 
terminates when one side or the other has been annihilated. 

Mathematically the problem may be stated as 



maximize ry(T) - px n (T) - qx 9 (T) with T unspecified 

Ht) 1 z 



subject to: 



dx i „ 

IT ' -«V 



dx. 

— = -(l-0)a 2 y 



= -b x - b x 
dt 11 22 



( 1 ) 



x^x^y £ 0 and 0 £ <j) £ 1, 



where all symbols are defined in the next section and with terminal 
states defined by (1) x^(T) = x 2^ ” 0 and (2) y(T) = 0. 

The terminal surface of the "realistic" (one-sided) game is 



seen to consist of five parts: 
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C 1 : x x (T) =0, x 2 (T) > 0, y (T) = 0 

C 2 : x^(T) = 0 before x 2 (T) = 0, y(T) > 0 
C 3 : x 1 (T) = 0 after x 2 (T) = 0, y(T) > 0 



C 4 : X]L (T) > 0, 


x 2 (T) = 0, 


/—V 

H 

II 

o 


C 5 : x 1 (T) > 0, 


x 2 (T) > 0, 


y (T) = 0. 



> 



3 . Notation , 

The symbols which are used in this paper are defined as 

follows : 

A = A(R,z) = [ z 2 (R-l) - R] / (z-1) 2 , 

B = B (R , z) = A(z-l) 2 /z 2 = [z 2 (R-l) - R]/z 2 , 
a l ,a 2 > ^ > l > ^2 = constant attrition rates, 

C. for i = 1,2, 3 ,4, 5 = the i— part of the terminal surface 

as defined in section 2., 

D(C_^) = domain of controllability for C_^, 

g(P°,R,z) = term in equation (57) of the locus of points for 
which P^ = P^, 

h(P°,R,z) = term in equation (58) for boundary surface between 

the regions from which optimal paths lead to and , 

p,q,r = utilities assigned to surviving and Y 

forces respectively , 

p.(t) for i = 1,2,3 = dual variable corresponding to x.(t) 

1 (x 3 (t) = y (t) ) , 1 

p? for i “ 1,2,3 = boundary condition for dual variable at 

1 T = 0, ' 
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P. for i = 1,2, 3,4, 5 = payoff associated with an extremal 

leading to C_^, 

P° = (x^jX^jYq) = point in the initial state space, 

R = a l b l y/ ^ a 2 b 2' > 5 

f ° °x , O O 

s = s(x 1 ,x 2 ) = b 1 x 1 + b 2 x 2 , 

t^ = time at which is annihilated, i.e. x^(t^) = 

t 2 = first time at which 2 b^x^(t 2 )x 2 + b 2 (x 2 ) 2 = a 2 y 2 (t 2 ) b ° r 
an extremal leading to , 

T = total time for the battle, 



v = v ( t ) = a p (t) - a p (t), 

. a l (b l P 2 + b 2 p) 

w = cosh /a b x (C. ) = -o- 7 — r 7 — x , 

2 2 14 p 2 “ a 2 b 2^ 



x^,x 2 ,y = average force 

V 

z = cosh /a^b^ ^(C^) = 
6 = a 1 p/(a 2 q) , 

0 = fraction of Y-fire 



strengths; with initial values 

fi < b i<i - y> R . { 
q ^ a i b i “ a 2 b 2^ R “ 

directed at , 



o 



r 



O 



X 



2 ’ 



x = "backwards time" from the end of the reduced game, i.e. 
the time remaining before the end of the reduced game. 



= "backwards time" of the first switch in tactics for 
extremals leading to C_^, 

x^ = T - t^ = length of time that Y fires at X 2 after X^ 
has been annihilated for extremals. 



4 . Solution Procedure and Extensions . 

Extremal paths (a path on which the necessary conditions for 
optimality are almost everywhere satisfied) may be obtained by routine 
application of Pontryagin’s maximum principle [16] (the original 



authors used equivalent conditions independently developed by Isaacs 
[13]). However, in a terminal control problem we would like to know 
the domain of controllability [7] for each terminal state so that 
tactics are determined in terms of the initial conditions of combat 
(and also possibly time). We define the domain of controllability 
for a given terminal state to be that subset of the initial state 
space from which extremals lead to the terminal state. Furthermore, 
additional considerations (as outlined below) are required when the 
domains of controllability corresponding to different terminal states 
overlap . 

The following procedure has been used to solve the above 

problem: 

(a) extremal control is determined by maximizing the Hamiltonian; 
since the state variables (force strengths) are non-negative, 

the control depends, in many cases, only on relationships between 
the dual variables (marginal return from destroying target) , 

(b) from each separate terminal state, the time history of the dual 
variables is obtained by a backward integration of the adjoint 
system of differential equations; for a square law attrition 
process, the adjoint equations are independent of the state 
variables , 

(c) for each terminal state the domain of controllability is deter- 
mined by forward integration of the state equations using the 
time history of extremal control developed in (b) ; changes in 
control with time (existence of transition surface) may have to 



be considered in this step. 



(d) the solution is determined at this point for regions of the 
initial state space which are covered by part of the domain of 
controllability for only one terminal state; one must also 
verify that the entire initial state space has been accounted 
for, since otherwise one may have overlooked some type of 
’’singular" surface or even a terminal state, 

(e) if domains of controllability overlap so that for a point of 
the initial state space contained in their intersection there is 
more than one extremal leading to the terminal surface, then one 
computes the payoff associated with each extremal; the optimal 
trajectory is selected from the extremals by comparing these 
values . 

It is noted that Isbell and Marlow [11] stopped at step (b) above. 

It seems appropriate to further discuss the situation when, 
for example, the domains of controllability corresponding to two 
different terminal states overlap. For points of the initial state 
space contained in the intersection of the two domains, there are two 
mutually exclusive possibilities: (1) for every point the payoff 

associated with the extremal leading to one terminal state always 
dominates that corresponding to the other, or (2) the payoff corre- 
sponding to extremals of one family does not always dominate that of 
the other. In the first case, the solution is simply determined: 
the optimal trajectories are the extremals corresponding to the more 
favorable payoff. In the second case, one must determine for what 
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parts of the region of intersection each alternative is better. For 
the problem at hand, it will be seen that such a region of intersec- 
tion may be divided into two subregions separated by a boundary surface. 

In each subregion the payoff corresponding to one family of extremals 
dominates that of the other. R. Isaacs (see pp. 132-141 of [14]) has 
referred to such a singular surface as a dispersal surface, since when 
the geometry of the optimal trajectories is considered, optimal paths 
only lead away from a dispersal surface. For the problem at hand, it 
will be seen that for a certain range of model parameters, two dis- 
persal surfaces are present in its solution. 

The complete solution to this problem is shown in Tables I 
and II with supporting details presented in the next section. When 
a^p ^ a 2 ^> the domains of controllability do not overlap so that the 
extremals are unique, and the extremal control turns out to be the 
optimal control. Hence, the solution procedure terminates at step (d) 
above. When a^q > a^p > the domains of controllability are shown 
in Table I as Case (2). For R - /R(R-l) £ 6 < 1 where 6 = (a p)/(a q), 
the domains of controllability don*t overlap except for that corresponding 
to part of the terminal surface, which overlaps several others. 

In section 5.2 below it is shown how optimal trajectories are deter- 
mined in the region of overlap and that extremals leading to can 

only be optimal for a^b^y^ 2 ^ + ^^2 X 2^* The resu ^ t: *- n & solution 

is shown in Table II as Case (a) . It should be noted that non-binding 
constraints, which define the regions in the initial state space for 
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TABLE I. Solution to Target Selection Problem - Fight to Finish 
Nonrestrictive assumption: R > 1, i.e. a^b^ > a 2^2 

Terminal State Optimal Contro l 



Case (1) : a 2 q £ a^p 



X l^l^ = ° 
C x / x 2 (T) > 0 

y(T) - o 



0*(t) = 



A for 0 £ t £ t 1 
^0 for t^ £ t £ T 



Domain of Controllability 

a 1 b 1 y Q 2 < s 2 + (R-l)(b 2 x 2 ) 2 
a l b l y o 2 S s2 " (b 2 x 2 )2 



' x l (t l ) = 0 
x 2 (T) = 0 

y(T) > o 



A x (t) > o 

c 5< x 2 (T) > 0 
^y(T) = 0 



Case (2) : a 2 q > a p 




x 1 (t 2 ) > 0 
x 2 (T) = 0 
y(T) = 0 



0*(t) = 



fl for 0 £ C £ t. 



t 0 for £ t £ T 



0* (t) = 1 for 0 £ t £ T 



Extremal Control 



Same as Case (1) 



Same as Case (1) 



0*(t) 



'1 for 0 £ t £ t_ 



0 for t 2 £ t £ T 



0*(t) = 0 for 0 £ t £ T 



0*(t) =■ 



1 for 0 £ t £ T - x. 



0 for T - t x £ t £ T 



a l b l y o 2 > s 2 + (R-D(b 2 x°) 2 



a lV ° 2 < s2 + (R-D(b : x °) 2 



^Vo 2 < s2 - (b 2 X 2 )2 



Definition of Times 

(a) tj is first t such that x^(t^) = ^ * 

(b) t 2 is first t such that 2b^x^(t 2 )x 2 + b 2 (x°) 2 - a 2 y 2 (t 2 ). 

^ a x (b 1 q-b 2 p) 

(c) Tj_ is determined by cosh /Zfo T l = T Ta^-a^)'* 



a l b i y o 2 < s 2 + (R-l)(b 2 x°) 2 

a l b l y 0 2 5s s 2 + B(b 2 x°) 2 

Same as Case (1) 

a^b^y Q 2 < s 2 + (R-l)(b 2 x 2 ) 2 
a l b i y o 2 5s Rts 2 - (b^^x®) 2 } 

a 1 b 1 y o 2 £ s 2 + R(R-l)(b 2 x°) 2 

a 1 b 1 y o 2 < s 2 + (R-l)(b 2 x°) 2 
a 1 b 1 y Q 2 < R{s 2 - 0>jX°) 2 } 

a i b i y o 2 * Rs2{1 " 7? ] 

a^b^y Q 2 < s 2 + (R-l)(b 2 x°) 2 
a 1 b 1 y Q 2 < s 2 + B(b 2 x°) 2 
a 1 b 1 y o 2 > s 2 + A(b 2 x°) 2 

a i b i y o 2 > Rs2{1 " 7 ? } 



Nonrestrictive assumption: R > 1, i.e. > a 2 b 2 



Case (a): A £ 0, i.e. R - /r(R-1) £ 6 < 1 



TABLE II. Target Selection Problem - Fight to Finish 

Solution for a 2 q > a^p and Various Values of Parameters 



A-ll 



Terminal State 



Optimal Control 




0*(t) =< 



0*(t) = . 



0*(t) = 



A 1 (t> > 0 



3 



(T) >0 0*(t) = ■ 



y(T) = o 



1 for 0 £ t £ t. 



0 for t 1 £ t £ T 



1 for 0 £ t £ t. 



0 for t 1 £ t £ T 



1 for 0 £ t £ t„ 



0 for t 2 £ t £ T 



r l for 0 £ t £ T - t. 



0 for T - £ t £ T 



Region of Initial Force Levels 



a l b i y o 2 < s2 + ( R-1 )( b 2 x 2^ 

a 1 b 1 y Q 2 £ s 2 + B(b 2 x 2 ) 2 



a lVo 2 > s 2 + (R-l) (b 0 x°) 2 



2 2 



a i b l y Q 2 * R ^ s2 “ (b-jX®) 2 } 

a 1 b 1 y o 2 £ s 2 + A(b 2 x°) 2 



a 1 b l y Q 2 > s 2 + A(b 2 x 2 ) 



°) 2 

r 

°\2 



a l b i y o 2 < s2 + fi ( b 2 X 2 ) 
a^b^y^ 2 > Rs 2 {1 - 1/z 2 } 



0*(t) = 0 for 0 £ t £ T 



Terminal State Optimal Control 
ase (b) : A > 0, i.e. 0 £ 6 < R - /R (R-l)" 



Same as Case (a) 



Same as Case (a) 



Same as Case (a) 



0*(t) = 0 for 0 £ t £ T 



a^b^y^ 2 £ Rs 2 {1 - l/z 2 } 

a by 2 < R{s 2 - (b x°) 2 } 
1 1 o 11 

Region of Initial Force Levels 



a 1 b 1 y Q 2 < s 2 + (R-l) (b 2 x 2 ) 2 

a i b l y o 2 ^ R s 2 - R {b 1 x°[z 2 ( R -l)+ R ]/(2 R ) + b 2 x 2 ) 2 /z 2 
for 0 £ x° < (b 2 x 2 )/(kb 1 > 
a^b^y^ 2 £ h(P° , R , z) for 0 £ x° £ kb.jX°/b 2 

Same as Case (a) 

a 1 b 1 y o 2 < s 2 + (R-l)(b 2 x°) 2 

a by 2 £ R{ s 2 - (b x°) 2 } 

1 1 o I i 

a^b^y^ £ h(P°,R,z) for 0 £ x° £ kb 1 x°/b 2 

a 1 b 1 y Q 2 < s 2 + (R-l)(b 2 x 2 ) 2 
a 1 b 1 y Q 2 < R{s 2 - (b^) 2 } 

a l b l y o 2 £ Rs 2 - R{b lX °[z 2 (R-l)+R]/(2R) + b 2 x 2 ) 2 /z 2 
for 0 £ x° < (b 2 x 2 )/(kb^) 



xplanation of Symbols 

(1) For t^ , t 2 and see Table I. 

(2) k = [z 2 - R(z-l) 2 ] / (2R) . 
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the various types of optimal control, have been deleted in constructing 

Case (a) of Table II from Case (2) of Table I. For 0 £ 6 < R - /r (R-1) , 

D(C^), the domain of controllability for the C^ part of the terminal 

surface, and D(C,-) have a non-empty intersection and similarly for 
D(C^) and D(C^). Hence, step (e) in our above solution procedure 
must be applied. In section 5.3 the determination of the two dispersal 
surfaces is shown. The resulting solution is shown in Table II as 
Case (b) . 

The above solution procedure may be easily extended to terminal 
control differential games (such as [19] in which the subsequently 
well-known necessary conditions [1] were not applied). Moreover, in 
two-sided problems (both the X-force and the Y-force are heterogen- 
eous and each is faced with a fire distribution problem affected by 
the other’s choice) this author has noted that not only may domains 
of controllability overlap with there being multiple extremals from a 
given point in the initial state space, but also the corresponding 
terminal states imply that the same player may either win or lose 
depending upon which extremal path is followed. This is illusory, 
however, since the player’s opponent may block his steering the course 
of battle to a winning end point by use of a non-extremal strategy. 

Hence, the player must always lose. 

We should note that in the existing theory of differential 
games [2], [14] it is assumed that the optimal strategies are pure. 

Since the Hamiltonian is separable (a function independent of the 
X-force ’s strategy variable(s) plus a function independent of the 
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Y-force's strategy variable(s)) for two-sided fire distribution 
problems in the Lanchester theory of combat as formulated by Isbell 
and Marlow [11] and Weiss (see pp. 94-95 of [18]), it may be shown 
that there do exist pure strategy solutions to such problems [8]. 
Thus, this paper ? s developments are applicable to this class of 
problems . 



5 . Development of Solution . 

The solution is actually derived for a "reduced" game (that 
portion of battle during which Y is faced with a choice problem) . 

We illustrate here for extremals to . It suffices to trace extre- 
mals up to t^ when x^(t^) = 0, since p* = 0 from then until the 
end of the game. The determination of the value, denoted by V(x^,X 2 >y)> 
of the reduced game, which is needed to determine the values of the 
adjoint variables on the terminal surface, and part of the solution 
originally obtained by Isbell and Marlow will not be repeated here 
although we shall outline the general steps. 

The Hamiltonian is 



H(t,x i ,p i ,0) = -{p 1 0a 1 y + p 2 (l-0)a 2 y + p^b^ + b^)}, (2) 



and the adjoint equations are 



dp l 

&T = b l P 3 ’ 
d Po 



dT = b 2 P 3 ’ 
dp^ 



dT = p i a i^ + P 2 (1 ' 0)a 2’ 



(3) 
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with 



p (t=t^) = unspecified 



/ \ 3 V 

P 2 (t=t l } = 



-q/b 1 x 2 
/b 2 x| - a 2 y z 



p 3 (t = ti) 



3V 

3y 



qa 2 y 

/bj /b 2 x^ - a 2 y z 



(4) 



The extremal control is determined by the maximum prin- 

ciple, i.e. maximum H(t,x.,p.,0), and we also have that 
O<0=£l 1 1 



H(t,x i ,p i ,0*) = 0. 



(5) 



Equations (4) and (5) may be combined to yield that fo^ extremals 
leading to we have 






-qa 2 /b^ x 2 (t=t 1 ) 



a, /b„x£ - a y z 



or 



1 2 2 






where 



( 6 ) 

( 7 ) 



p l <t-t l > 



lim p (t) 

*^1 

tSt l 



Obtaining a solution to this problem is simplified by the 
following considerations. Let t = t^ - t and define 



v(t) = a 2 P 2 (x) - a^p^ ( t) , 



( 8 ) 
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then we have 

dT = (a i b l" a 2 b 2 )p 3 ( - T) ’ (9) 

with 

v (x=0) = a 2 p 2 (x=0) - a^^O), (10) 

and where (up until the first shift of tactics) 



P 3 (t) = p 2 (x=0) coshfv^a^b^ + (l-?>)a 2 b 2 x} 

0a p (x=O)+(l-0)a 2 p 2 (x=0) 

sinh{/0a b + (l-0)a b x} . (11) 

vfya^ + (l-0)a 2 b 2 11 Z Z 

It is easily seen that p^(t) > 0 for a ll T > and thus 

> 0 for all t, (12) 



when > a 2 b 2 (9). As determined by the maximim principle and 

use of (2) and (8) , the extremal control is given by 

fl for v(t) > 0 

0*(t) =< 

^0 for v(t) < 0. (13) 

It is easy to show that it is impossible for v(t) = 0 over any finite 
interval of time, and hence the possibility for any singular solution 
[15] to this problem is excluded. (The reader should note that the 
term "singular surface" is used in a slightly different sense by 
Isaacs (see pp . 132-134 and pp . 156-158 of [14]) and in the control 
theory literature (see p. 502 of [10] or p. 4 of [15]).) 
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By the symmetry of the problem it suffices to assume that 
a^b^ > a 2^2’ an ^ t ^ len ec J ua ti° ns (7), (8), (12) and (13) imply that 
extremals can actually lead to C^. Furthermore, for a ^b^ > 
it is readily shown that no extremals lead to (i.e. D(C^) 

void) and that extremals can only lead to when > 0 for 

t < T. We shall sketch the proof that no extremals for which x 2 ^ t ^ = ® 
for t < T can lead to C^ . By arguments similar to those used above 
for and using the value of the reduced game V^^x^y) and (2) 

and (5) with t = 0 and 0* = 0, it is readily shown that v(x=0 + ) = 0 
where x is the backwards time from the end of the reduce game, i.e. 
x^(x=0) = 0, and v(x=0 + ) is a one-sided limit through positive 
values. Considering (12), this implies that v(x) > 0 for x > 0, 
and hence by (13) is never fired at so that cannot be reached. 

The special case in which is reached but x 2 ^ t ^ > ^ for t < T 

is discussed in section 5.2. 

In the next three subsections we present additional details 
of solution development: (1) determination of domains of controlla- 

bility, (2) entry to , and (3) determination of dispersal surfaces. 
This material represents the extension of the original work of Isbell 
and Marlow [11] . 

5.1. Determination of Domains of Controllability . 

One contribution of this note is to show how to determine the 
domains of controllability. There are two cases to consider. 
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Case (1) a^q £ a^p 

These determinations are routine and consist in combining the 
time history of the extremal control, the non-negativity requirements 
of the state variables, and the generalized square law 

u 2 (t 1 ) - u 2 (t 2 ) = {0a 1 b 1 + (l-0)a 2 b 2 > (y 2 ( t^) - y 2 (t 2 >), (14) 

where 0(t) = const. in t^ 3C t i t^ and u(t) = b^x^(t) + b^x^Ct). 

Case (2) a^q > a^p 

The condition for entry to C^ is as before. There are two 
subcases for entry to C^. First we consider the extremals for which 

/l for 0 £ t < T - t 
0* ( t) = -/ 

^0 for T - t ± * t 5 : T. (15) 

Combining the above extremal control with the generalized square law 
(14) , we obtain 

R(b 2 x 2 (T))2+ 2Rb 1 b 2 x 1 (T-x 1 )x 2 (T) + (b^ (T-^) ) 2 

+ 2(l-R)b 1 b 2 x 1 (T-T 1 )x° + a i b iy Q Z - s 2 + (1-R)(b 2 x°) 2 = 0. (16) 

A backwards integration of the state equations of 0 £ t £ with 
x 2^ t= ^ ~ X 2^ T ^ and y(T=0) ” 0 yields that 

b 2 X 2 = b 2 X 2 ^ Z + b 1 x 1 ( T “ T 1 ) (z-1) > (I 7 ) 

since X 2 (t=T-x^) = x° . It should be noted that in deriving (17) we 
have assumed that x^(T-t^) > 0. (16) and (17) may be combined to yield 
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(b 2 x 2 (T)-b 2 x°) 2 = (a 1 b 1 y Q 2 -s 2 )/A, (18) 

and hence we require that the right-hand side of (18) be non-negative 
in order that C,_ be reached. It may be shown that A > 0 leads to 
a contradiction to a result (32) derived below, and hence A £ 0 for 
extremals corresponding to (15) to lead to C,_ . This corresponds to 
R - A(R-l) < 6 < 1, since A = [z 2 (R-l) - R]/z 2 and z = (R-6)/(R-l). 
When A < 0, solution of (18) for x 2 (T) anc * tbe requirement that 
x^(T) > 0 leads to 

a by 2 > s 2 + A(b x°) 2 . (19) 

1 1 o l l 

Combination of the expression for x 2 (T) with (17) and the require- 
ment that x^(T-t^) > 0 leads to 

a^b^y^ 2 < s 2 + (b 2 x°) 2 A(z-l) 2 /z 2 . (20) 

Next, we consider the extremals for which 

0>*(t) = 0 for 0 £ t < T. (21) 



It is readily shown that 



y(t) = y Q cosh/a^b^ t 



(b^x° + b 2 X 2^ 



s inh /a 2 b 2 t , (22) 



so that the boundary, which occurs when y(T) = 0 for T = t^, 
between this case and the previous one is easily seen to be given by 



There is a flaw in this argument. The final solution, however, is not 
affected by this. See Appendix F for a further elaboration of this and 
other subtle points. 
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(b,x“+b x“) 2 

y o 2 [cosh/a 2 b 2 t^ 2 = — { [cosh/a^ x^ 2 - 1}, (23) 

where the i-time of the switch, determined from integration of the 
adjoint equations and the maximum principle, is given by 



z = cosh/a^b^ = 



(b x q - b 2 p) 
q (a l b l ” a 2 b 2 ) 



(24) 



Noting that 0* = 0 for the entire battle when T < and re-arrang- 

ing, we obtain the result shown in Table I. Combination of the extremal 



control (21) with (14) also yields a quadratic equation for ^^(T) 
which may be solved to yield 



b 2 X 2 (T) = -b l x l + “ a 2 b 2 y o i! * ( 25 ) 

Thus, the payoff for extremals on which (21) holds is given by 

P 5 = {(b 1 x°) - R - r - <S - ~ - / s 2 ~ (a 1 b 1 y o z )/R). (26) 

The requirement that X£(T) > 0 leads to the result shown in Table I. 
When this latter condition or (19) is violated, then it is not possible 
for x^(T) > 0 when 0*(t) = 0 for 0 £ t £ Hence, we must 

investigate the possibility that = ® (C^ is entered), and 

this is done in the next section. 

Finally, we consider extremals leading to C^. As done above, 
we may combine the extremal control with (14) and the requirement that 
x^(T) >0 to obtain the first result shown in Table I. Similarly, 



y(t^) ^ 0 implies that 
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a l b l y o 2 a (b l X l )2 + 2b l b 2 X l X 2' (27) 

Also, the payoff for extremals leading to C is given by 

P 3 — /s 2 + (R-l)(b x°) z - a b y z . (28) 

1 /R b 2 2 2 1 1 0 

It remains to develop conditions which distinguish between 
optimal paths leading to and . (Thus, strictly speaking the 

result for shown in Table I Case (2) is for the region where 

extremals of D(C^) yield a larger payoff than extremals to when 

the two fields of extremals overlap.) We do this for A < 0, i.e. 
tP~ < R/ (R-l) , since different arguments are required to determine 
optimal trajectories when A > 0 (see section 5.3). It will be shown 
that optimal paths can only lead to when the time remaining in the 

battle after destruction of the forces exceeds the maximum time 

to fire at for extremals reaching . 

Let T i = ^ denote the length of time that Y fires at 

X^ after X^ has been annihilated. Then optimal paths can only 

reach when ^ t^C,.) = where t^(C,-) t ^ ie "backwards 

time” of the switch in tactics for extremals. First, we shall 

show that extremals from P° with x° > 0 leading to yield a 

better payoff than those leading to for a^b^y^ ** “ l/z 2 } 

and 7?~ < R/ (R-l) . Thus, optimal paths can only lead to from 

initial points for which a_b y 2 > Rs 2 {1 - l/z^}. However, for such 

1 1 o 

points the extremal strategy is to use ?>*(t=0) = 1 and to continue 
this until either the switching surface is encountered or X^ is 
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annihilated, i.e. 0*(t) = 1 for 0 £ t £ minimum (T-x^, T-x^) . 

Hence, optimal paths can only reach when x^ ^ x^. 

Thus, it remains to sketch the proof that extremals from P° 
with x° > 0 leading to yield a larger payoff than those leading 

to for a^b^y^ 2 ^ Rs 2 {l - 1/z 2 } and z 2 < R/(R-1). Setting 

a^b^y^ 2 = Rs 2 {1 - 1/z 2 } in the expressions for and P^ (equa- 

tions (26) and (28) , respectively) , it is easily shown that P > P 

3P 1 aP 5 5 1 

for z 2 < R/ (R-l) . Then since ^ ^ 0, it is readily seen 

y o y o 

that P c > P- for all y such that a-b.y 2 £ Rs 2 {1 - 1/z 2 } when 
51 ^o 1 l'o 

A < 0. 

Combining the extremal control for with (14) , we have 



that 



y2(t i } = dr {a i b i y o 2 ‘ (b i x i )2 ' 2b iVl x 2 } 



(29) 



11 



Also, since 0*(t) = 0 for t^ j£ t £ T, integration of the state 
equations (1) yields that 



y (t) = y(t=t 1 )cosh/a 2 b 2 (t-t^) - ~ sinh/a^ (t— t^) - (30) 



Since y(t=T) =0, we have that 



tanh/a 2 b 2 



y (t =t l) /^2 



V 



(31) 



Since the hyperbolic cosine is a strictly increasing function of its 
argument, x^ ^ x^ implies that cosh/a^b^ x^ ^ cosh/a^b^ x^ ^ 1. 
Combination of this with (29) and (31) , use of well-known identities 
for the hyperbolic functions, and some algebraic manipulation yields that 



A-22 



, 2 . °s2 . fZ 2 (R-l) - Ri 

a l b l y o * (b 1 x 1 +b 2 x 2 ) 2 + { ? 1 



(b 



x°) : 
2 2 ) 



(32) 



For 1 < z 2 i. R/(R-1), it is easily seen that (32) implies (27) so 
that this latter condition need not be considered further. 



5.2. Entry to Terminal State . 

Terminal state C. is reached whenever X„ can be annihi- 
4 2 

lated by Y during the battle's last phase of length x^. The bound- 
ary conditions on the terminal surface are 

x 1 (T) > 0 P 1 (x=0) = -p 

x (T) = 0 p o (x=0) = p° unspecified 

z z z 

y(T) = 0 p^x^O) = p°, unspecified (33) 

We also have that 0*(t=T) = 0*(x=O) = 0, and hence by (33) and the 

transversality condition (5) it follows that p° = 0. It should be 
noted that p° remains unspecified on the terminal surface. It will 
be shown that p° is determined by conditions for extremals to reach 
C^. Recalling the extremal control at t = T and (13), we also must 
have that 

a l P//a 2 < ( _p 2^ ’ ( 34 ) 

Since v(x=0) < 0 and > 0 by (12), there will be a 

dx 

switch in tactics in “backwards time 11 at t = T-(C.) = T n . (In this 

14 1 

section the notation x^ refers to extremals leading to when it 

is not denoted otherwise.) Thus, the extremal control is given by 
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0*(t) = 



T. for 0 s£ t £ t„ 



u 0 for t 2 s: t £ T, 



(35) 



where t,, = T - and t 2 is such that 



2b 1 x 1 (t 2 )x° + t> 2 ( x 2 ^ 2 = a oy 2 ( c o) 



2 v 2 ' 



(36) 



From a backwards integration of the adjoint equations and the maximum 
principle, the time of the switch in tactics is given by 



a (bp + bp) 

w - cosh^ ^ -(^b— • 



(37) 



We observe that from what region of the initial state space extremals 
lead to is determined by the value assigned to p° . In other 

words, the value assigned to p° determines what initial force levels 
x° are to be totally destroyed. It seems reasonable to argue that 
(-p°) cannot exceed q, since otherwise we would implicitly be valuing 
survivors greater than stated for the problem. A more precise 
argument, of course, is to compare payoffs (i.e. and , as 

done in section 5.1) for extremals leading to and C,_ from the 

same initial point P° corresponding to p° > q, but this has not 
proven to be computationally tractable. Thus, recalling (34), we must 
have 

a l p / a 2 < (-P^) * <1- 



(38) 
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As done previously, we combine the extremal control (35) with 
(14) to obtain 

( b l>^l ( 1 2 ) ) 2 _ 2(R-l)b 2 x°b 1 x 1 (t 2 ) + a 1 h 1 y Q 2 - s 2 - (R-l)(b 2 x°) 2 = 0, 



which may be solved for to Y^ld 



b l X l^ t 2^ = b 2 x 2^ R_1 ^ + ^ + R ( R_1 ) ( b 2 x p' i “ a l b l y 0 Z ’ (40) 



whence the result shown in Table I by requiring that the quantity under 
the radical sign is non-negative. The other entries for in Table 

I correspond to other parts of the terminal surface being impossible 
to reach. 

It remains to show that optimal paths can only lead to 
from P° when 



a 1 b 1 y Q 2 £ s 2 + A(b 0 x°) 2 , 



2 2 ' 



(41) 



where A = A(R,z) and z = cosh/a^b^ t^(C^). A backwards integration 
of the state euqations (1) for 0 £ t £ x^(C^) = T i x^(t=0) = 

x l(t 2 ) and x 2 (t= 0) = 0 yields that 



V2 

X l <t 2 ) ‘ b, (w-1) • 



<42) 



since x 2^ t=t i^ = x° and x^(t=t^) = Combining (39) and (42), 

we obtain 



s 2 + A(R,w) (b 2 x°) 2 - a^b 1 y Q 2 = 0. 



(43) 



(39) 
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For constant R, A may be considered to be a function of w. Con- 
sidering the first two derivatives of A(w) and lim A(w) , it is 

W"*» 

readily seen that for 1 ^ w s: A(w) has a global maximum at 

w* = R/(R-1) with A(w*) = R(R-l) > 0. This latter result implies 
that for P° satisfying (41) the quantity under the radical sign in 
(40) is non-negative (i.e. the corresponding entry in Table I is 
satisfied) . It should be observed that w(-p°) is a strictly in- 



creasing function of (-p°) 


with 






( a i P^ 

o l r - 

w - p 2 = a 1 
2 a 2 ' 


and w(-p°=q) 


R-6 . R 

Z R-l * R-l " 


(44) 



Since for 1 £ w £ w* , A(w) is a strictly increasing function; then 

a lP o 

for £ (-p ) ic q, A(w(-p ) ) is a strictly increasing function 

a 2 ^ ^ 

whence (41). Also, combination of (37), (40) and (42) yields that 




(p°» 



a l p{b 2 X 2 (R_1) + /s(pU » R)} 
a 2 /S(P°,R) 



(45) 



where 

S(P°,R) = s 2 + R(R-l) (b 2 x°) 2 - a 1 b 1 y Q 2 * (46) 

Equation (45) shows the dependence of dual variable values on 
the terminal surface upon the initial force levels P° of an extremal 
It seems appropriate to observe that for a^p/a^ £ (~p°(P°)) ^ q we 
have that 0 -£ ^(C.) ^ t. (C c ) and also ^(C.) = t.. (P°) . The value 
implicitly assigned to survivors through p^ is adjusted so that 

the forces are just annihilated at t = T. Let us also note that 
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by construction we have made v(t=t^ (P°) ) =0 in our determination 
of (-p 2 (P°)). The above should make clear the reason that p° must 
be unspecified on : for extremals corresponding to a given value 

of x^(T) and leading to there are an infinite number of different 

values for t^(P°) for these extremals that result in this x^(T). 

5.3. Determination of Dispersal Surfaces . 

It seems appropriate to note here some relationships between 
the inequalities which define the domains of controllability in Table 



I. It is always true that 

s 2 + B(b 2 x°) 2 £ s 2 + (R-l) (b^x^) 2 . (47) 

For R - /R(R-l) £ 6 < 1, the following hold 

s 2 + A(b 2 x°) 2 £ s 2 + (R-l) (b^) 2 , (48) 

Rs 2 {1 - 1/z 2 } £ s 2 + (R-l)(b 2 x°) 2 , (49) 

s 2 + A(b 2 x°) 2 £ s 2 + B(b 2 x°) 2 , (50) 

Rs 2 { 1 - 1/z 2 } £ s 2 + B(b 2 x°) 2 . (51) 



In (51) , there is strict inequality for x° > 0 and R - /R(R-l) < 6 < 1. 
For 0s{<R- vR (R-l) , we also have 

s 2 + A(b 2 x°) 2 > s 2 + B(b 2 x°) 2 , (52) 

Rs 2 { 1 - l/z 2 } > s 2 + B(b 2 x 2 ) 2 . 



(53) 
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The proofs of the above are straight-forward and omitted. 

When 0 j£ 5 < R — /R(R-l) , the above inequalities (52) and 

(53) imply that D(C^) and D(C,_) for extremals on which $*(t) = 0 

for 0 £ t £ T overlap (i.e. there is a region in the initial state 

space from which extremals lead to both and C^) and similarly 

for D(C-) and D(C.) when a-b-.y 2 £ s 2 + A(b n x°) 2 in D(C.). 
i 4 llo 22 4 

Hence, we must consider step (e) of section 4. f s solution procedure. 

First, we consider the determination of the dispersal surface 
which separates regions of the initial state space from which optimal 
paths lead to and C,_ . Let us observe that for z 2 > R/(R-1) 

points exist for which 

s 2 + B(b 2 x°) 2 ic a b y 2 £ Rs 2 {1 - 1/z 2 }. (54) 

Setting a.,b y 2 = s 2 + B(b 0 x°) 2 in the expressions (26) and (28) for 
llo 22 

P,_ and P^ respectively, it is easily shown that it is optimal to 
go to C,_ from such an initial point. The dispersal surface is the 
locus of points in the initial state space for which P^ equals P 
aid extremals lead to both and C,_ on both sides of this sur- 

face. Equating (26) and (28), the locus of points for which P^ = P^ 
is given by 

a by 2 = Rs 2 - R{b x°[z 2 (R-l) + R]/(2R) + b x°} 2 /z 2 . (55) 

llo 11 22 

Furthermore, a dispersal surface actually exists, since it is 



readily shown that 
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Rs 2 - R{b i x°[z 2 (R-l) + R]/(2R) + b 2 x°} 2 /z 2 < Rs 2 {1 - 1/z 2 }, (56) 

when z 2 > R/ (R-l) . (56) and an above result mean that on both sides 

of the candidate dispersal surface extremals lead to both C and CL. 

3P 1 3P 5 1 3 

Observing that ^ ^ 0 , it is readily seen that P_ ^ P_ for 

3y o 3y 0 15 

all P° such that a^b^y^ ^ Rs 2 " R{b^x^ [ z 2 (R-l) + R]/(2R) + b 2 X°} 2 /z 2 . 
Since we also must have a^b^y ^ 2 < R^s 2 ~ (b^x^) 2 } for extremals 
leading to C^. , the above dispersal surface only exists for 0 :£ x° < 
2Rb 2 x°/{b 1 [z 2 -R(z-l) 2 ]}. 

Based on the above results there may also be a dispersal sur- 
face which separates optimal paths leading to and for 

0 < x° < b^x^[z 2 - R(z-l) 2 ] / (2Rb ) . Its determination is similar to 

the above, although the details are messier and haven 1 t been completely 
worked out . Let 

a lVo = g(P°,R,z), (57)- 



be the locus of points for which P^ = P^ . Then, the boundary, which 
separates optimal paths leading to and , is given by 

a l b l y o =h ( p °> R > z )> (58) 

where 



h(P°,R,z) 



/s 2 + B(b 2 x °) 2 
g(P°,R,z) 

' s 2 + A(b 2 x °) 2 



for g(P°,R,z) < s 2 + B(b 2 x °) 2 

for s 2 + B(b 0 x °) 2 .< g(P°,R,z) s£ s 2 + A(b 2 x °) 2 

for g(P°,R,z) > s 2 + A(b 2 x°) 2 . ( 59 ) 



It should be noted that this 
only when extremals to both 
= g(P°,R,z) . 



boundary surface is a dispersal surface 

CL and C, lie on both sides of a-.b_.y2 
14 1 1 o 
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6. Structure of the Optimal Allocation Policies . 

For square-law attrition it may be shown that the allocation 
of fraction of fire is always 0 or 1. In section 5 we have dis- 
cussed the fact that the coefficient of the control variable f> in 
the Hamiltonian HCtjX^p^,#) cannot be equal to zero over any finite 
interval of time. This excludes the possibility of a singular control 
[15], and maximization of the Hamiltonian, which is a linear function 
of 0, yields that all fire is concentrated on one target type. This 
is not surprising, since our model assumes complete and instantaneous 
information [2] and that fire may be immediately shifted to a new 
target once the old one has been destroyed [5], [18]. 

With reference to Tables I and II, the condition that a^b^ > 
a^b^ may be interpreted to mean that there is more long range return 
for Y to engage X^, i.e. more Y T s will survive if this is done. 
Hence, when Y wins, he always engages X^ ! s while they are available. 
The condition a^p < a^q means that at the end of battle there is 
greater payoff per unit time per Y soldier to engage X^ not con- 
sidering X^ T s greater attrition effect against Y (short term gain 
at end of battle) . 

By the maximum principle and the well-known interpretation of 
the dual variables [1], Y always allocates his fire entirely to the 
target type yielding the greatest marginal return. However, marginal 
return evolves differently in winning or losing causes. When Y loses, 
he may switch from firing at X^ entirely to firing at X^ entirely 
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to firing at entirely before the force has been annihilated. 

This happens when Y assigns utility to survivors of force type X^ 
in excess of their kill rate against Y as compared to force type 
X^, and X^ is abundant enough not to be destroyed before the battle 
ends. Under these circumstances, whenever the X^ forces are annihi- 
lated, this occurs at the very end of battle. 

In this way, we see that tactics may depend on force levels. 
Moreover, the time at which the ranking of target priorities switches 
itself may depend upon force levels. We also see that Y f s target 
priorities only switch with time in a losing case. This has occurred 
since a boundary condition at t = T on one of the dual variables is 
dependent upon values of the state variables by a transversality con- 
dition. It may be shown that the structure of optimal allocation 
policies is different for the prescribed duration battle. 



A-31 



REFERENCES 



[1] L. Berkovitz, "Necessary Conditions for Optimal Strategies in 
a Class of Differential Games and Control Problems," SIAM J. 
Control, 5, 1-24 (1967). 

[2] , "A Survey of Differential Games," Mathematical 

ThcoAy ofi ContAol, A. Balakrishnan and L. Neustadt (Ed.), 

Academic Press, New York (1967). 

[3] L. Berkovitz and M. Dresher, "A Game Theory Analysis of Tactical 
Air War," OpenattonA RcACOAch, 7, 599-620 (1959). 

[4] , "Allocation of Two Types of Aircraft in Tactical Air 

War: A Game Theoretic Analysis," OpcAatio VIA ReA COAch , 8, 

694-706 (1960). 

[5] H. Brackney, "The Dynamics of Military Combat," OpCAationA 
RcACOAch, 7, 30-44 (1959). 

[6] L. Dolansky, "Present State of the Lanchester Theory of Combat," 
OpcAationA RcacoacH, 11, 344-358 (1964). 

[7] 0. Elgerd, ContAol SiJAtcmA ThcoAy, McGraw-Hill, New York (1967). 

[8] A. Friedman, "Existence of Value and of Saddle Points for 
Differential Games of Survival," JouAnal ofa Vi^CACntial Equations , 
7, 111-125 (1970). 

[9] D. Fulkerson and S. Johnson, "A Tactical Air Game," OpCAatconA 
ReAcaAch, 5, 704-712 (1957). 

[10] Y. C. Ho, "Review of the Book VifificAential Gama by R. Isaacs," 
IEEE Tacwa. on Automatic ContAol, Vol. AC-10, 501-503 (1965). 

[11] J. Isbell and W. Marlow, "Methods of Mathematical Tactics," 
LoglAtlcA PapeAA , No. 14, The George Washington University 
Logistics Research Project, September 1956. 

[12] , "Attrition Games," Naval RcACOAch LogiAticA 
duaAteAly, 3, 71-94 (1956). 

[13] R. Isaacs, "Differential Games-The Basic Principles of the 
Solution Process," The RAND Corporation, RM 1411, Vol. Ill (1954). 

[14] , Vifj&eACntslal GameA, John Wiley & Sons, Inc., New York 



(1965) . 



A-32 



[15] C. D. Johnson and J. E. Gibson, "Singular Solutions in Problems 
of Optimal Control," IEEE Trans. on Automatic- Control, Vol. AC-8, 
4-15 (1963). 

[16] L. Pontryagin, Y. Boltyanski, R. Gamkrelidze and E. Mishchenko, 
The Mathematical Theory 0 Optimal Processes , Interscience 
Publishers, Inc., New York (1962). 

[17] S. Sternberg, "Allocation Strategies," Chapter 2 of Appendix D 
in Development ofi Models fior Detente Systems Planning, S. Bonder 
and R. Farrell (Eds.), Report No. SRL 2147 TR 70-2 (U), Systems 
Research Laboratory, The University of Michigan (1970). 

[18] H. Weiss, "Lanchester-Type Models of Warfare," PtOC. F trst 
International Confi. Operational Res., Oxford (1957). 

[19] , "Some Differential Games of Tactical Interest and 

the Value of a Supporting Weapon System," Operations Research, 

1 , 180-196 (1959). 



Appendix B. Lanchester-Type Models of Warfare and Optimal Control. 



1 . Introduction . 

The optimization of the dynamics of combat is studied 
through a sequence of idealized models by use of mathematical theory 
of optimal control. The models are for combat over a period of 
time described by Lanchester-type equations with a choice of tactics 
available to one side and subject to change with time. A sequence 
of models is examined for each of two types of choice situation: 

selection of target type for engagement, 
regulation of firing rate. 

Optimal tactics are discussed with reference to the influence of 
combatant objectives, termination conditions of conflict, type of 
attrition process, variable attrition rates, and limited ammunition. 
Implications for intelligence, command and control systems, and 
human decision making are pointed out. The use of such optimal 
control models for guiding extensions to differential games is 



discussed . 
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In this appendix we examine the structure of optimal allocation 
policies for tactical situations describable by Lanchester-type 
equations of warfare. We hope to provide insight into such questions 
as 

(1) How should targets be selected? 

(2) Do target priorities change with time? 

(3) Does the number of target types effect the selection? 

(4) Do battle termination circumstances effect the optional 
allocation policies? 

(5) How does the nature of the attrition process effect 
target selection? 

(6) What is the effect of ammunition constraints? 

(7) How does the uncertainty and confusion of combat effect 
the optimal selection rules? 

Our theory of tactical allocation is developed through the examination 
of a sequence of simplified models. These combat models are too simple 
to be taken literally but should be interpreted as indicating general 
principles to serve as hypotheses for subsequent computer simulation 
studies or field experimentation. 
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In 1964 Dolansky [ 10 ] noted that the Lanchester theory of 
combat was insufficiently developed in the area of target selection 
for combat between heterogeneous forces (optimal control/differential 
games). Even the two references cited by him, Weiss [32] and Isbell 
and Marlow [16], have been subsequently extended by this author [26], 
[28]. Since Dolansky T s article, no further examples have been published 
in the literature except for the ones in Isaacs T book [15]. This pre- 
vious work had never systematically investigated the dependence of 
tactics upon model form. 

We examine several idealized combat situations described by 
Lanches ter-type equations over a period of time with choices of tactics 
available to one side and subject to change with time. We consider 
models for two types of choice situations 

(1) selection of target type, 

(2) regulation of firing rate. 

These problems are solved by the mathematical theory of optimal control. 
A further elaboration on solution development is to be found in our 
report [25]. 

In the first sequence of models we examine the effect on 
optimal target selection of the following factors: objectives of the 

combatants, termination conditions of the conflict, number of target 
types, some special cases of time dependent attrition rates, and type 
of attrition process. We then examine a sequence of models to see how 
ammunition limitations effect firing rates. Next we discuss two-sided 
extensions of such problems but point out the value of studying one- 
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sided problems as considered in this appendix. Finally, various 
implications of the models are discussed. 

2 . Target Selection . 

We begin by considering the simplest situation of target 
selection: combat between an X-force of two force types (for example, 

riflemen and grenadiers) and a homogeneous Y-force (for example, 
riflemen only) . This situation is shown diagrammatically below. 




It is the objective of the Y-force commander to maximize his survi- 
vors at the end of battle at time T and minimize those of his 
opponent (considering weighting factors p, q and r) . This is 
accomplished through his choice of the fraction of fire, 0, directed 
at X . We may study this idealized tactical allocation problem in 
two types of scenarios: (1) a battle lasting a specified time, T 

or (2) a battle lasting until one side or the other is totally 
annihilated . 

2.1. Battle of Prescribed Duration. 



Mathematically the problem may be stated as 
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maximize ry(T) - px (T) - qx 0 (T) with T specified 
(t) 1 1 



subject to: 



0 a x y 



. 

dt 
dx 2 

dT = ' (1_ ^ )a 2 y 



= - b x - b x 

dt Vl b 2 2 



^,x 2 , y ^ 0 and 0 £ 0 £ 1, 



where 

p,q, and r are weighting factors assigned to surviving forces, 
x^,x 2 and y are average force strengths, 
a^,a 2 ,b^ and are constant attrition rates, 

and 0 is fraction of Y-fire directed at X^. 

This problem may be solved by routine application of Pontryagin 
maximum principle* [22]. The solution is shown in Table I. In this 
present analysis we have not considered those subcases when a state 
variable is reduced to zero. It may be shown that the structure of 
the optimal allocation policies is not fundamentally altered by such 
an occurrence. 

With reference to Table I, we see that two characteristics of 
the optimal allocation policies for this particular prescribed dura- 
tion battle are: 

(1) concentration of all fire on one target type, 

(2) independence of allocation from force levels. 

*We employ the equivalent version commonly used in this country 



(see p. 108 of [9] ) . 



Battle of Pr scribed Duration with Constant Attrition Rates 
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We shall later see that when there are more than two target types in 
this scenario, the solution possesses these same characteristics (even 
when the attrition rates change over time) . Both these characteris- 
tics, however, are consequences of the assumed model form. 

The first characteristic, concentration of effort on one 
alternative, is a consequence of the "square-law” attrition process 
for the X-forces. (We shall refer to attrition as being a "square-law” 
process when the casualty rate is proportional to the number of enemy firers 
and as being a "linear-law" process when it is proportional to the 
product of the number of enemy firers and remaining targets.) It may 
be shown that this makes the existence of a singular control [18] 
impossible, and hence the optimal allocation policies are extreme 
points in the control variable space. 

There is, however, a very simple principle which underlies the 
above mathematical formalities: concentration of effort when constant 

marginal returns are obtained from the alternatives and the total effort 
is limited. Constant marginal effect over time per unit of weapon system 
is a property of the "square-law" attrition process, for let us consider 
the X^-force attrition (when 0=1) 



dx i 

dt 




f 

rate 


of 


casualties produced per 


y 


' a i 


unit 


of 


Y-force weapon system^ 



Thus there is a constant (or non-diminishing) marginal effect over 
time. This should be contrasted with the situation for a "linear-law" 
attrition of the X^-forces 




Vi = 



f y 

rate of casualties produced per 
unit of Y-force weapon system^ 



In this case we have diminishing effects over time from allocating a 
unit of Y-force weapon system against X^, and a division of total 
effort (i.e., fraction of fire) may be called for. B. Koopman's 1953 
article [21] contains an excellent discussion of such principles ' 

which underlie such an optimization problem. Presently, we shall 
verify these heuristic arguments in a mathematically precise fashion 
when we consider a dynamic model, which considers the interaction of 
forces over time, in which both X-force target types undergo "linear- 
law” attrition. This fundamental difference in the structure of opti- 
mal allocation policies based on the nature of target attrition makes 
the determination of the appropriate attrition process an essential 
task of analysis. 

The second characteristic, target selection independent of 
force levels, is due to the combination of the "square-law" attrition 
process for the X-force types and the fixed battle length, T. It 
is seen that for the battle of prescribed duration target selection 
depends only on the attrition rates of the various force types and 
relative weights assigned to surviving force types. This is not surprising > 
since it is easily seen that the adjoint differential equations are independen 
of the state variables, and the values of the dual variables at the 
end of battle t = T are independent of force strengths. It is 
recalled that a dual variable represents the rate of change of the 
payoff (battle outcome as measured by the value of surviving forces 
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at t = T) with respect to a particular state variable [3]. Thus, 

if if V = ry (T) - px^T) - qx 2 (T), then p^t) = — (t) , etc. 

Hence, the boundary conditions are given for the dual variables at 

3V 

the end of the battle t = T as = — (t=T) = -p 5 P 2 (t=T) = 

-q,P 3 (t=T) = r. 

It seems appropriate to discuss further the interpretation of 
the solution shown in Table I. From the above definition of the dual 
variables , 



a- L P 1 (t) 



r 

effect on outcome per 
unit time for engaging 




kill rate of 
Y against 



r 

effect on outcome per 
unit of destroyed 



Hence, the condition a^p < a^q means that at the end of the battle 
(recall that p^(t=T) = -p, etc.) there is greater effect on battle 
outcome (as measured by value of survivors) per unit time per soldier 
for Y to engage (short term gain at the end of battle) . The 

value of the dual variable, for example, p^(t) reflects both the 
value assigned X^-force survivors and the dynamic interaction of 
forces over time through the Lanchester-type equations. Hence, it 
also accounts for the effectiveness of X^ against Y. The quantity 
a^b^ may be interpreted as representing the instantaneous rate of 
destruction of the X^-force kill rate against the Y-force per unit 
of Y-force. Then a 1 b 1 > a 2 b 2 means that there is greater strategic 
value for engaging the X^-force, i.e., more long range return. Thus, 
case A of Table I corresponds to where there is both more long range 
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and also short range return for engaging X^. Case B corresponds to 
more short term gain at the end of the battle for engaging » but 
more long range return for engaging X^. It is easily shown that 
case A results when Y values surviving X-forces in direct propor- 
tion to their kill rate against the Y-force, i.e., p/q = b^/b^. 

A switch in tactics (target priority) is seen to occur for this model 
only when value is not assigned to survivors of a target-type in 
proportion to their destructive capability (kill rate) . 

The maximum principle may be interpreted as saying that a 
target type from several alternatives is engaged when such an engage- 
ment yields the greatest favorable effect on battle outcome per unit time, 
turns out, tho i, that the evolution of target engagement return is 
dependent upon the scenario chosen for the study of the problem. This 
is clearly seen when we examine the "fight to the finish." This is a 
special case of a terminal control battle (the combat ends only when 
the course of battle has been steered to a prescribed end state) and 
is chosen for mathematical convenience. 

2.2. Terminal Control Battle . 

We consider the similar problem of 

maximize ry(T) - px (T) - qx (T) with T unspecified 
0 (t) 1 2 



dt 

dx 2 

dT 



= - 0 a x y 

= - (1-0) a 2 y 



= 

dt 



Vi 



b 2*2 



subject to: 
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V 


* 2 ,y ^ 0, 


0 




and with 


terminal 


states 


defined by (1) 


x x (T) = 


X 2 


(T) = 0 j 


and 


(2) y(T) = 0. 




The terminal surface of 


this problem 


is 


seen to 


consist 


of five parts: 


V 


x x (T) = 


0, 


x 2 (T) > 


0, 


x 3 (T) 


= 0 




V 


x x (T) = 


0 


before 


X 


2 (t) « 0 


, x 3 (T) 


> 0 


V 


x 1 (T) = 


0 


after 


X 2 


(T) = 0, 


x 3 (T) : 


> 0 


V 


x x (T) > 


0, 


x 2 (T) - 


o. 


x 3 (T) 


= 0 




V 


x x (T) > 


0, 


x 2 (T) > 


0, 


x 3 (T) 


= 0 





The above problem was first studied by Isbell and Marlow [16], 
but a complete solution was first obtained by us in a previous paper 
[26]. We have also extended the solution principles to terminal 
control differential games, for which we have outlined a general 
solution procedure and have used it to solve the supporting weapon 
system game of H. K. Weiss [28]. The solution to this problem appears 
complex (see Table I of [26]) but may be described in a particularly 
simple fashion (for the nonrestrictive assumption that a^b^ > a 2^2^* 
In contrast to the battle of prescribed duration, the optimal 
target engagement may depend on initial force levels. When Y wins, 
he engages until depletion before X^. When Y loses, he may 

switch from firing at X^ entirely to firing at X^ entirely before 
the X^ force has been annihilated. This happens when survivors of 
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force-type are assigned utility in excess of their kill rate as 

compared with force-type X^, and certain relationships hold between 
initial force strengths. This dependence of the optimal allocation 
on initial strengths has been caused by the fact that values of dual 
variables at t = T are dependent upon values of the state variables. 
This happens in terminal control attrition problems where a value of 
a state variable is specified at the terminal surface (and hence the 
value of the corresponding dual variable is unspecified but may be 
determined from the transversality condition H(t = T,x,p,0) = 0, 
where H(t,x,p,0) denotes the Hamiltonian). 

2.3. Prescribed Duration Battle with Several Target Types . 

We have considered th r * st two problems in orde< to contrast 
the effect of the battle termination conditions upon the structure of 
the optimal allocation policies. Another factor that we can examine 
is the number of target types. For the prescribed duration battle, 
certain facets which tended to be obscured in the scenario with two 
target types are brought into sharp focus. In a companion paper [30] 
we present the supporting analysis details, and in this section we 
summarize solution properties in order that they may be contrasted 
with those of other scenarios in this first sequence of target selec- 
tion problems. 

We consider the following prescribed duration problem: 
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n 



maximize vy( 

0 ± (O 

subject to: 



' - L 

i= 


w.x. (T) 
1 1 1 


dx . 

l 

dt 


- 0 i a i y 


= 

dt 


n 

- 1 b.x 

i-1 1 1 




n 


^ 0, 


and £ 




i=l 



where all symbols are used in the same sense as previously. The 
solution to this problem turns out to be a generalization of that in 
section 2.1. However, certain aspects receive greater emphasis to 
provide us with a deeper understanding of the phenomena under study. 

In particular, we considered two subcases, denoted as case A and case 
B, in the solution to the previous problem of section 2.1. (see Table 
I). When there are several target types, the generalization of the 
subcases which we must distinguish is as follows: 

Case A, enemy survivors valued in direct proportion to their 
kill rate against Y-force, 

Case B, enemy survivors not valued in direct proportion to 
their kill rate against Y-force. 

In the first instance, case A, it may be shown that target 
priorities keep their same relative ranking over time. Assuming for 
the moment that no force type is totally annihilated during the course 
of battle, the optimal allocation policy is 
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0*(t) 



6 . . for 0 < t £ T , 
ij 



where 6 .. is the Kronecker delta and is equal to 1 if i = j and 

zero otherwise, and j is the index such that a.b. = max (a 1 b 1 ,...,a b ). 

J J linn 

If the highest priority target type is exterminated during such a battle, 
then fire is merely shifted to the next highest priority target. Hence, 
when one values enemy survivors in proportion to their kill rate against 
you, i.e., w^ = kb^ for i = l,...,n, the optimal tactic is to con- 
centrate all fire on a single target type until it is entirely destroyed. 
The sole criterion for target selection in this instance is the quantity 

a i^i 5 w hich may be interpreted to be the rate of destruction of enemy 

t h 

attrition capability for his i force type (see section 2.1.). 

In case B, it may be shown that there will be at least one 
switch of target priorities if the battle lasts long enough. Again 
assuming that no force type is totally annihilated during the course 
of battle, it may be shown that for x^ ^ T the optimal allocation 
policy is given by 



0*(t) = 6. for 0 < t £ T, 
l in 



where j is the index such that a.w. = a.[-p.(t=T)] = max (a.w, ,. . . ,a w ) 

3 3 3 3 1 1 n n 

and we have arranged our indexing so that this is the last index, i.e., 
j = n, p_.(t) is t ^ ie dual variable corresponding to the state variable 
x_. , and t^, the "backwards" time x = T - t (measured from the end 
of battle) of the first switch in target selection, will be given below. 



The battle must last longer than x^ for a switch in target priorities 



B-15 



to occur. Assuming that it does, we now state what conditions are 
necessary for a change in target selection and the "backwards’ 1 time 
at which the change occurs, x^. 

In developing the solution to this problem we work backwards 
from the end of battle, t = T. Let k be the index of the target 
type to which fire is first shifted in "backwards" time, x. Then it 

may be shown that necessary conditions for fire to be switched to the 

th b k w k 

k target type are that a. b > a b and — > — , i.e., we shift 

k k n n b w 

n n 

fire to a target type which causes attrition in a greater proportion 
than the ratio of values placed upon survivor from the target type 
which yields the greatest direct return at the end of battle. The 
target type to which fire is shifted has index k determined by 



R. = min (R 1 , . . . ,R 1 ) , 

k R.>0 1 n_1 

a .b . >a b 
11 n n 



where 



R. 

1 



a . (b . w -b w . ) 
l i n n l 



a .b .-a b 



for i = 



ii n n 

The ’backwards" time of switch of fire to the 
determined from the transcendental equation 



1 ,.. 

k th 



,n-l . 
target. 



V 



is 



w 

n 



cosh /a b x , 
n n 1 




sinh/a b 
n n 



a, (b, w -b w, ) 
k k n n k 

a. b. -a b 
k k n n 



This is seen to be a generalization from the problem with two X-force 



target types. 
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Barring the extinction of a force type, the optimal allocation 
policy for x^ < T but T 2 ^ T is given by 

0*(t) = for 0 s; t < T - x. 

1 lk 1 

0*(t) = 6. for T - T-. £ t £ T, 

1 in 1 

where x^ is determined by (1) and a similar expression exists for 

x 2 9 the "backwards" time of the second change. A similar solution 

with two changes in the optimal allocation policy exists when t 2 ^ ^ 

but x^ > T. The number of switches in target engagement depends on 

the length of battle, T, and as the battle progresses forward in 

time fire is always shifted to a target type for which both a.b. 
b . 

and — are smaller. If the battle lasts long enough, then during 
w . 

1 

the initial stages all fire is concentrated on the X-force type lor 

b. 

which both the quantities a.b. and — are larger than any other. 

i 

Although the details differ, the optimal allocation policies 
are seen to have the same structure as that for just two target types 
(see section 2.1.): 

(1) concentration of all fire on one target type, 

(2) independence of allocation from force levels. 

The addition of more target types has not changed the nature of the 
problem. This problem 1 s explicit solution is a generalization of that 
with two X-force target types. 

It is of interest to ask whether the optimal tactic will always 
be to concentrate fire on only one target type (bang-bang optimal 
control) . The answer to this question turns out to be "no" as 
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consideration of a "linear-law" attrition process for the X-force 
target types will show in section 2.5. An extensive heuristic dis- 
cussion of the principles involved has been given in section 2.1. 

2.4. Some Special Cases of Time Dependent Attrition Rates . 

In the previous idealizations of combat that we have considered 
above, we have assumed that all the Lanchester attrition-rate coeffi- 
cients were constants. In reality, this coefficient depends on 
numerous factors some of which are as follows: hit probabilities, 

weapon system projectile-target lethality characteristics, rates of 
fire, rate of target acquisition. These factors themselves may be 
range dependent or change over time. S. Bonder [ 6 ], [ 7 ] has devel- 
oped explicit formulas for relating the Lanchester attrition-rate 
coefficient to weapon system performance characteristics such as those 
mentioned above. 

Thus, it seems appropriate to examine idealized combat situations 
in which the attrition rates are time dependent. Solutions have been 
obtained to variable-coefficient Lanches ter-type equations for a square- 
law attrition process between two homogeneous forces under very general 
circumstances [29]. However, such solutions are, in most instances, 
so complicated that they cannot be applied to corresponding optimal 
control problems to yield easily interpretable analytic results which 
provide insight into the structure of the optimal allocation policies. 
There is a class of variable coefficient Lanches ter-type equations 
(combat between two homogeneous forces when the attrition rates are 
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variable provided that their quotient is a constant) which possess a 
solution no more complicated than the solution to the constant 
coefficient case [27]. Hence, when one considers optimal control 
problems for Lanchester combat between forces with time dependent 
attrition rates, there are some special instances of practical 
interest (reflecting the physical situation in which two weapon 
systems cause attrition in a proportional fashion at all times) that 
are not much more mathematically complicated than the idealized 
situations we have considered in sections 2.1 and 2.3 above. Again, 
we shall merely summarize results, leaving the details for a companion 
paper [30]. 

We consider the following prescribed Station battle: 

maximize ry(T) - px n (T) - qx Q (T) with T specified 
0 (t) 

subject to: 



x i’ x 2 ,y ^ ° 



dx^ 

dt 

dx r 



= - 0 a 1 (t)y 



dT" = -(l~ 0 )a 2 (t)y 



d£ 

dt 



b 1 (t)x 1 - b 2 (t)x 2 



and 0 £ 0 m 1 . 



We assume that both X-force weapon systems are such that 



bf (t ) = h(t) and b^(t) = h(t) 



In the special case when the Y-force values surviving 
X-force types in direct proportion to their kill rate against the 
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Y-force, i.e., p/q = /k^ = b^tj/b^t) = b ± (t=T) /b 2 (t=T) , the 

optimal control law takes a particularly simple form 



In this instance target selection depends only on the product of 
attrition rates which may be interpreted as the rate of destruction 
of enemy kill rate. All fire is concentrated on one of the target 
types depending on which target type has the larger attrition-rate 
product. Target priority is subject to change over time as the rank- 
ing of the target types on this decision criterion changes. It is 
conceivable that the optimal tactic may be to shift fire from one 
target type to the other several times over the course of battle with 
the duration of battle not having any effect. Observe that no 
assumptions at all have been made on the Y-force attrition rates 
against and X ^ , i.e., a^(t) and a^(t). 



a 9 (t) = k h(t). This means that not only is the ratio of the X-force 
z a 0 



weapon system attrition rates against the Y-force constant but also 
the ratio of the Y-force effectiveness against each of the two 
X-force types. Furthermore, all four attrition rates have the same 
time dependence except for constant factors. The solution is shown 
in Table II. In this special case (under the assumptions noted above), 
we see that the structure of the optimal allocation policies when the 
attrition rates are variable is essentially identical to that when 




At this point we further assume that a^(t) = k h(t) and 




2 



Battle of Prescribed Duration with Variable Attrition Rates 
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they are constant. Only the time scale has been transformed (in [27] 
we first made this type of observation). 

The extensions to more than two target types in each of the 
two cases considered above possess essentially the same solution 
details as those noted here. These extensions are similar to those 
we considered in section 2.3. 

2.5. Selection of Targets Undergoing "Linear -Law" Attrition . 

So far the state equations have described combat according 
to the Lanchester square law in which attrition of a target type is 
proportional to the number of each force type firing at it. Weiss 
[31] has given a thorough discussion of the conditions which lead to 
this. These conditions include that n each unit is informed about the 
location of the remaining opposing units so that when a target is 
destroyed, fire may be immediately shifted to a new target." It is 
noted that the control theory models which we have considered so far 
have implicitly assumed perfect information. 

Another model for attrition is the Lanchester linear law in 
which the average decrease of a target type is proportional to the 
product of the average number of targets remaining and the number of 
each force type firing at it. Such a dependence can arise under two 
general circumstances: (1) fire is uniformly distributed over a 

constant target area ("area fire") or (2) the mean time of target 
acquisition is much larger than target destruction time and is inversely 
proportional to target density. The first circumstance corresponds 
to the simplest case of partial information . Again quoting Weiss [31], 
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we assume that units are informed about the general areas in which 
opposing units are located, but are not informed about the consequences 
of their own fire. Thus, we see that we may account for some changes 
in the information set by modifying the description of combat. Brackney 
[ 8 ] has shown that "aimed fire" may lead to linear-law attrition when target 
acquisition times are considered, and are as postulated above. 

Thus, we consider the following problem in which the X-forces 1 
attrition obeys a linear-law process and the Y-forces* attrition obeys a 
square-law process: 

minimize ry(T) - px., (T) - qx ? (T) with T specified 
0(t) 

subject to: 



x i ,x 2 *y ^ 0 
* 

In a future paper we shall present the analysis details upon 
which our present summary is based (see also pp 91-105 of [25]). Since 
the state and adjoint equations do not readily yield an analytic 
solution, we have not been able to obtain explicit expressions for 
certain model parameters. However, we can still discuss all the quali- 
tative characteristics of the structure of the optimal allocation 
policies . 

* 

See Appendix D. 



_ 

dt 

^2 = 
dt 

= 

dt 



- 03-^y 
-(l-0)a 2 x 2 y 



- b lXl - b 2 x 2 



and 0 £ 0 £ 1. 
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There is a fundamental difference between the solution to 
this problem and those considered previously: the optimal allocation, 

0*, may be other than 0 or 1. In contrast to the previous 
problems, the optimal allocation policy does not have to be an extreme 
point of the control variable space at all times. We may have a 
singular solution [18] for which the necessary condition of maximizing 
the Hamiltonian (with respect to the control variable) does not pro- 
vide us with a well-defined expression for the extremal control. We 
shall call the part of an optimal trajectory on which the maximum 
principle cannot be used to determine the control a singular subarc. 
Then the term M singular solution" will be used to refer to any optimal 
trajectory which contains one or more singular subarcs. 

Singular solutions can only occur when the Hamiltonian 

(denoted as H) is a linear function of the control variables. When 

3 H 

this is so, then if — = 0 for a finite interval of time (or, 

00 

another way to say this, the coefficient of 0 vanishes identically 

for a finite interval of time) , the maximum principle does not 

3H 

determine the control. Observe that when — = 0 and H is a linear 

00 

function of 0, all feasible values of 0 are optimal. All problems, 
however, for which the Hamiltonian is a linear function of the control 
variables do not have singular subarcs in their solution. For example, 
it may be shown that such a singular control is impossible for the 
problems considered above in sections 2.1 through 2.4, since it is 



impossible for 



dt 



3H 



l 9 0J 



3 H 

= 0 for a finite interval of time when — = 0. 

00 
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For the problem at hand, it may be shown that a necessary condition 
for a singular subarc to yield the maximum return [20] is satisfied. 

The optimal battle trajectories are constructed by working 
backwards from all possible end points of this idealized battle. Con- 
sideration is given to both the optimal control at the end of battle 
and also how the variables upon which it depends vary over time. 

Based upon such considerations, it may be shown that there are three 
cases to be considered: 

b 

Case (a) — = — , 

q b 2 

b 

Case (b) > — , 

q b 2 

b 

Case (c) < - — . 

q b 2 

We consider Case (a) first. The solution for this case is 

shown in Figure 1. Even though explicit expressions have not been 

obtained for certain model parameters, the dependence of the optimal 

control upon these quantities can still be qualitatively discussed. 

It may be shown that the optimal control depends on the -state variables 

x ^ and x^ (and also the attrition coefficients) in each '‘decision 

region." Above the line a 2 ^i x i = a 2^2*2 5 denoted as L, the optimal 

control 0* =0 is used until this line is encountered. When L is 

a 2 

reached, the singular control 0* = — , which keeps the trajec- 

a l + a 2 

tory on L, is used until the end of the battle at t = T. That 



Case (a) 
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Figure I. Optimal allocation for linear-law attrition. 
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portion of an optimal trajectory which lies on L (for a finite 

interval of time) is a singular subarc. The above type of solution 

holds for arbitrary initial values of and : x^(t=0) = x^ 

and X2(t=0) = x^ . The time history of the optimal control is traced 

for two particular initial force ratios denoted as points A and D 

X 1 a 2 b 2 

in Figure 1. At point D, — > — — and 0* = 1 is used until the 

u a. b 
x^ 11 

line L is encountered at point E. 

The solution for Case (b) is shown diagrammatic ally in Figure 
2. It is similar to the preceding case except that another line, L’ 
with equation a^px^ = a 2^ x 2 5 Pl a Y s a central role besides the 
singular ’’surface" denoted as L. The final segment of that part of 
any optimal trajec+*o^y which lies below L 1 consists of the use of 
the optimal control 0* = 1 for the final stages of battle. Such is 
the case for the optimal trajectories denoted as (2), (3), and (4). 

Any optimal trajectory which lies entirely above L f , such as (1), 
has a corresponding optimal control of 0*(t) = 0 for 0 £ t £ T, 
whereas a similar remark holds for any one that lies entirely below 
L, such as (5). Case (c) is symmetric to Case (b) . 

As has been noted previously (see sections 2.1 and 2.3), the 
structure of the optimal allocation policies in these tactical alloca- 
tion problems is dependent upon how the Y force values the surviving 

X-force types relative to their kill rate against the Y force, 
b^ 

Case (a) : = - — above is when Y assigns utility to surviving 

q b 2 



X-force types in exact proportion to their destructive capability 
against Y. In this case, the optimal target selection tactic depends 



Case (b) 
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x 



Figure 2. Optimal allocation for linear - law attrition. 
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only upon the state of the system as is seen with reference to Figure 
1. The optimal tactic is to use 0*(t) = 0 above the line L with 
equation a ^^2. X l = a 2^2 X 2* ^he ^ ne ^ a l so represents an "equili- 
brium" trajectory which the system follows whenever this line is 
reached . 



Case (b) : ^ > — is when Y assigns a greater value to 

0. t>2 

surviving X^ f s than in proportion to their kill rate against Y 
relative to that of . Again, the optimal tactic depends upon the 
state of the system, only this dependence itself depends upon the 
"time phase" of the fixed length battle. There is a x^ such that 
for 0 < x ^ x^» where x is "backwards time" measured from the end 
of battle x = T - t, the optimal tactic depends upon the state of 
the system relative to the line L ! with equation a^px^ = a >^ x 2 
(see Figure 2). Below L f the optimal tactic is to use 0*(x) = 1. 
For x^ £ x ^ T, the optimal tactic is to use 0*(x) = 0 above the 
line L with equation a ^^^ x ^ “ a 2^2 X 2 # We reca H t ^ iat the part of 
an optimal battle trajectory that coincides with L is a singular 
subarc. It may be shown that the "crossover time," x , depends 
upon the particular battle trajectory under consideration. 

Based on our examination of the above problem, we see that the 
structure of the optimal allocation policies for targets which undergo 
a linear-law attrition process has the following characteristics: 



(1) possible division of fire between target types, 

(2) dependence of allocation upon force levels. 
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These characteristics should be contrasted to those previously observed 
with respect to a square-law attrition process (see sections 2.1 and 
2.3). When there is a linear-law attrition process for target types, 
we may have other than 0 or 1 as the optimal allocation policy. 
Also, the allocation depends upon the force levels of target types. 

We have attempted to explain this structure of the optimal allocation 
policies in terms of the nature of the attrition process in section 
2 . 1 . 

3 . Regulation of Firing Rate . 

In this section we will examine a sequence of models of 
increasing complexity for which the effect of ammunition limitations 
on firing rate(fire discipline) will be explored. In each case, we 
consider two homogeneous forces engaged in combat described by a square 
law. Our results for these problems are of a more preliminary nature 
than those reported above for target selection. 

3.1. Battle of Prescribed Duration with Constant Attrition Rates . 

We consider the problem 



maximize px(T) - qy(T) with T specified 
0(t) 

. dx 

subject to: - - a^y 

dy 

dir = ' ? va z x 



dz 

dT = 



:,y ;> 0, 0 < 0 < 1, z(t=0) = 0 and z(t=T) £ A < vT = 



= v 



dt, 
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where v is the maximum firing rate of each X unit. It should be 
noted that the nature of the attrition coefficients and a^ is 

different, since a^ has incorporated in it a firing rate. 

We have assumed that each X combatant has a limited supply 
of ammunition (his "basic load") , denoted by A. There is no redis- 
tribution of ammunition during the course of battle, i.e., when a 
casualty occurs, his ammunition is lost to the other combatants. We 
further assume that the ammunition supply is such that a combatant 
could not fire at his maximum firing rate for the prescribed duration 
of the battle, for when A ^ vT it is easily seen that the optimal 
strategy is to fire at the maximum possible rate, 0*(t) = 1 for 
0 £ t £ T. 

The optimal regulation of firing rate turns out to be 

A 

0*(t) = 1 for 0 ^ t ^ T 1 where T^ = — , 

0*(t) = 0 for T^ < t £ T. 

This was determined as follows. The Hamiltonian is given by 

H(t,x i ,p i , ) = 0v(p 3 ~P 2 a 2 x) - P^y, 

where p^(t) is the dual variable corresponding to the i*"* 1 state 
variable (we denote x as x^, y as x^, z as x^) . Maximization 
of the Hamiltonian leads to the following control law 

/0 for (~P 2 (t) )a^x < X* 
l 1 for (-p 2 (t))a 2 x > X* > 0, 



0*(t) = 
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where p^(t) = - The adjoint differential equations are given 

by 



F 1 

dt 


SH 

9x 0 va 2 p 2 


with 


P 1 (t=T) 


= P 


dp 2 _ 
dt 


8H . 

8y a l P l 


with 


P 2 (t=T) 


= - q 


P 3 (t) 


= constant 


with 


P 3 (t=T) 


unspecified 



To develop the desired solution we work backwards from the end of 
battle. Hence, we introduce the "backwards time" variable x = T - t 
and consider a backwards integration of the state and adjoint differ- 
ential equations from the fixed end of the battle, t = T. Thus, 
dp i 

= - 0 va^P 2 > etc. It is easy to show that p^(x), x(x), and 

y(x) are non-decreasing functions of x (regardless of the value of 
0) with p^(x=0) = p, x(x=0) = x g , and y(x=0) = y g . Similarly, 
P 2 (x) is a strictly decreasing function of x. Hence, Q(x) = 
(-P 2 (x))a^x is a strictly increasing function of x with an initial 
value of Q(x=0) = qa^x^. Thus, p^ must be negative, and 0*(x) 
never switches back to 0 once it becomes 1. 

The optimal tactic for this model is disturbing, since it is 
not intuitively appealing to fire at one's maximum firing rate until 
one runs out of ammunition and to spend the final stages of battle 
without ammunition. However, after a little reflection we realize 
that it is this model's assumption of constant attrition rates which 
has negated the holding of one's fire as an optimal tactic. Hence, 
we are led to consider other models for further insight. 
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3.2. Battle of Prescribed Duration with Time Varying Kill Rates . 
We consider the problem 



maximize px(T) - qy(T) with T specified 

0(t> 

dx 

subject to: -j^- = - a^tjy 

S = ' 0 va 2 (t)x 



dz 



x,y ^0, 0 < 0 £ 1, z(t=0) = 0 and z(t=T) £ A < vt. 

The precise nature of a^(t) and a^Ct) will depend upon the specific s< 
under consideration. It seems reasonable to assume that in many real 
world situations a^(t) and a^Ct) would be monotonically increasing 
functions of time, e.g., two forces closing with one another. All the 
previous solution steps remain the same except for the effect of a^(t) 
and a^Ct) increasing with time. This may change the nature of the 
solution markedly, although the optimal control is still bang-bang. 

One can show, in fact, that a singular solution is impossible for this 
problem. The quantity Q(t) = (-p 2 ( T ) )&£ ( T ) X ( T ) is no longer guaranteed 
to be a strictly increasing function of x , since a 2 ^ x ^ is strictly 
decreasing (but positive) . This allows the possiblity that the optimal 
tactic may be to hold one’s fire and conserve ammunition in the early 
stages of battle so that 0*(t=T) =1 at the end of battle. 
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The way in which ammunition is conserved depends on the 

specific nature of a^(t) and a^(t). As we have pointed out above, 

under special circumstances to solution to variable coefficient 

Lanches ter- type equations (and also the adjoint system of equations) 

is no more complicated than the constant coefficient case. If we 
va 2 (x) a 2 

assume that — r = , then there are relatively simple expressions 

a., (t J k 

1 a i 

for P 2 ^ t ) and x(t). For example, when 0=1 for 0 £ t £ t^, 
we have 



P 2 ( t ) = " q cosh 6 (t) 




0 (t) , 



(T 

where 0 (t) = /k k h(x)dT. However, even such relatively 

a l a 2 J 0 

simple expressions for these quantities has not led to a general 
statement about under what circumstances one holds his fire even 
though this can be readily numerically determined for any particular 
set of parameter values. A question of practical importance to be 
answered by further investigation along these lines is, "For what types 
of attrition-rate range dependencies is it the optimal tactic to hold 
one's fire until an enemy gets closer due to an ammunition constraint." 

R. Isaacs has studied some similar problems in his book 
Differential Games [15] and has explored some aspects much deeper than 
presented here. Isaacs tried to resolve the problem of shooting up all 
of one's ammunition before the end of the battle by modifying the pay- 
off. Another approach might be to consider a terminal control problem. 
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3.3. Fight to the Finish with Limited Ammunition . 

We consider briefly the constant coefficient problem of section 
3.1 only with T unspecified and with terminal states defined by 
(1) x (T) = 0 and (2) y(T) = 0. In this case, the optimal tactic 
is dependent upon the force levels, since the ammunition constraint 
is not binding for 



It should be noted that the above condition also guarantees that X 
will win. Furthermore, the X forces in order to win are required 
to have enough ammunition to fire at their maximum rate during the 
entire duration of the battle. Hence, we see that concentration of 
forces reduces the ammunition requirement per man, since the length 
of battle is determined by the initial numbers of forces committed 
to battle. 

3.4. A Two-Sided Extension. 



of the optimal regulation of firing rate for the above problems with 
a two-sided version, even though we do not attempt to solve 
the latter at this time. Thus, we consider the problem 




It seems appropriate to contrast qualitatively the structure 



maximize minimize px(T) - qy(T) with T specified 
0(t) ifl (t) 
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subject to: 



dx 

dt ” 

iz = 

dt 



- ^ a 1 v 1 y 

- 0 a 2^2 x 



du 

dl = ^2 




1 



x ,y ^ 0, 0^0, * 1 , u(t=0) 

v(t-0) 



0, u (t=T) ^ A 2 < v^T, 
0, v (t=T) iC A 1 < 



Unlike the previous one-sided version of this problem, it is now 
possible to have 0*(t=T) = 1 with limited ammunition* This possibility 
has arisen since the Y forces may hold their fire during the early 
stages of engagement. Questions now arise as to the advantage of 
delivering the first shot, e.g., is there a time lag before fire is 
returned?, and we move into the realm of games of timing studied at 
RAND [ 19 ] . 



4 . Extensions to Differential Games . 

Even though it is certainly true that combat is an environ- 
ment of conflicting interests in which the potential actions of both 
friendly and enemy forces must be considered, there is much to be 
learned from one-sided dynamic optimization models. We view these 
simplified idealizations presented here as ''building blocks" 

for more sophisticated scenarios. We feel, however, that an under- 
standing of the structure of optimal tactics for these initial models 
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is essential before one continues his examination of a sequence of 
models of greater and greater complexity. Hence, it seems appropriate 
to review the intimate connection between optimal control theory and 
differential games. 

It has been stated that optimal control problems may be 
viewed as one-sided differential games for which the roles of all but 
one of the competing players have been suppressed [3]. A concise dis- 
cussion of the inter-relationships between these two subjects is 
contained in Y. C. Ho’s [14] excellent review of Isaacs book [15] (see 
also Chapter 9 in [9]). 

When one recalls the equivalence of variational problems and 
partial differential equations of the first order (first pointed cut 
by C. Jacobi nearly 140 years ago [17]), the relationship between 
optimal control and differential games may be viewed as follows. In 
an optimal control problems we are seeking the solution to the Hamilton- 
Jacobi-Bellman equation for the optimal value function, denoted as S 
(sometimes referred to as Hamilton’s characteristic function in the 
calculus of variations literature [23]) 

— + maximum H(t,x ,t— ,0) = 0 

0 t / v 1 0 X . 

0(t) 1 

with appropriate boundary conditions. In a differential game we seek 
the solution to the Bellman-Isaacs equation [13] 

8 S 3S 

*r 1- maximum minimum H(t,x.,T = 0. 

at ,/\ i ox . ^ 

0(t) lj>(t) 1 
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Moreover, this approach may also be extended to when stochastic effects 
are present as has been done by S. Dreyfus [12] and others [11], [33]. 

It also seems appropriate to mention the relationship of 
dynamic programming to these techniques. Consideration of the equation 
satisfied by the optimal value function points out clearly an important 
aspect of dynamic programming, its being a discrete approximation 
technique for solving variational problems [12]. It is, however, a 
dual approach which generates an optimal trajectory as an envelope 
of tangents rather than as a sequence of points [1]. The value of 
these continuous models lies in their mathematical tractability which 
many times allows us to develop "closed-form" solutions. This should 
be contrasted with dynamic programming models for which general solu- 
tions are rarely obtained, and one must be satisfied with generating 
a numerical solution for a particular set of parameter values. 

In this case, it is difficult (if not impossible) to see the structure 
of optimal allocation policies and its dependence upon model form 
without a parametric analysis of model output. 

We have previously pointed out [26] that the existing theory of 
differential games is only applicable to problems with pure strategy 
solutions. The structure of two-sided fire distribution problems 
in the Lanchester theory of combat as formulated by H. Weiss [31] leads 
to such a result. However, when defensive capabilities were considered 
in the attrition process in a tactical air war game extensively studied 
at RAND, the resulting model did not possess a solution in pure 
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strategies [2], [4], [5]. Thus, there are limits to the applicability 
of such variational models. 

We have, therefore, used these optimal control problems to 
study many aspects of corresponding two-sided variational problems: 
the effect of different boundary conditions, devising solution pro- 
cedures, study of singular behavior, differences in the structure of 
optimal allocation policies for various model forms. Most solution 
aspects of the one-sided problem are present in the two-sided one. 

There are some significant differences, however. In solving the support- 
ing weapon system game of H. K. Weiss [32], we have encountered solu- 
tion behavior unique to terminal control attrition games: there may 

^xist a domain of controllability for a given terminal state but entry 
to this state may be n blockable n by the "losing" player [28]. In other 
words, there is a path determined by the necessary conditions leading 
from each point in a region of the initial state space to a terminal 
state, but the "losing" player may use a strategy other than his 
extremal strategy for this path to actually win. We made use of our 
knowledge of a related optimal control problem [26] to solve this 
differential game. 

5 . Implications of Models . 

It seems appropriate to briefly discuss the general implica- 
tions in the following areas of the models examined in this appendix: 

(1) optimal tactical allocation, 

(2) intelligence, 
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(3) command and control systems, 

(4) human decision making. 

Our discussion of these areas is not mutually exclusive. 

Of interest to the military tactician is whether target 
selection rules evolve dynamically during the course of battle. Are 
target priorities static or do they evolve dynamically with the course 
of battle? With respect to optimal control models, this may be 
mathematically stated as whether there are transition (switching) 
surfaces in the solution. We have seen in the idealized and simplified 
models studied here that target priorities do change. This is related 
to the evolution of marginal return of target destruction (value of dual 
variable). We have seen that this evolution depends on the goals of 
the combatants (utility assigned to surviving force types at the end 
of the battle) and also the conditions which terminate the battle. In 
the terminal control problem studied here, a shift in target priorities 
is present only in a losing case, whereas in a fixed duration battle 
such a switch is independent of winning or losing but depends only on 
weapon system capabilities and the prescribed duration of battle. 

Even though these models assume complete and instantaneous 
information, it appears that some inferences may be made for cases 
where uncertainty is present. In the terminal control case, we saw 
that selection of tactics depends on a knowledge of the enemy’s strength 
and capabilities, since the terminal state of combat must be determined 
before optimal strategies can be. For a battle of prescribed duration, 
e.g., fighting a delaying action in a regrograde movement to protect 
the withdrawal of troops, tactics depend only on enemy and friendly 
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capabilities and length of combat, not the initial force levels. For 
such cases the estimate of combat length is critical, since changes in 
target priorities are determined relative to the end of the engagement. 

Schreiber [24] has proposed an idealized and simple, but yet 
illuminating, way of quantitatively showing the value of intelligence 
and command control capabilities. He introduces the concept of "command 
efficiency," which is measured by the fraction of the enemy's destroyed 
units from which fire has been redirected. The effect of poor intelli- 
gence and poor capabilities for redirecting fire from destroyed targets 
is to produce "overkill." Schreiber ! s equations for combat involved 
this fraction called "command efficiency," and they reduce to Lanchester- 
type equations for area fire when the fraction is 0 and aimed fire 
for a value of 1. We have seen that the optimal tactics are quite 
different for these two cases. When intelligence and command control 
systems are very efficient, the optimal tactic is seen always to be 
concentration of fire on a specific target type. When capability for 
redirection of fire from destroyed targets is poor (either through damage 
assessment or constraints on new target acquisition) , the optimal tactic 
may be to allocate fire in a proportional fashion over target types in 
a way that holds the ratios of target density in each target area to 
be constant. Another implication is that supporting weapon systems 
(e.g., artillery) concentrate fire on selected point targets, but that it 
sometimes is best to allocate fire proportionately over various area targets 
these models suggest that the tactics of target engagement may vary 
with command and control capabilities. 
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These models also show the importance of intelligence in de- 
vising the "best 1 * tactics in combat. Intelligence on enemy weapon 
system capabilities (kill rates including target acquisition rates) 
and potential length of engagement play a central part. We also have 
seen that for fights to the finish and linear law attrition cases 
intelligence on enemy force levels is also required. For artillery 
fire support missions against various troop concentrations, knowledge 
of troop densities is essential in the assignment of target priorities. 
Particularly dense concentrations where the initial kill potential is 
high are seen to be cases where the optimal tactic is to concentrate 
fire on one target for awhile. 

Another argument for the concentration of forces is seen to 
emerge from the study of these simplified models. When ammunition is 
limited, a concentration of forces has the effect of counter -balancing 
this constraint. For example, in a fire fight numerical superiority 
could mean that the enemy force level would be reduced such that he 
would disengage in time before the friendly ammunition restriction 
became critical. 

These models may be interpreted to show the value of human 
judgment in combat. They indicate, as does common sense and experience, 
that in battle a commander must use his judgment to ascertain to what 
end can the course of battle be steered so that he may devise his 
strategy accordingly. The demonstrated sensitivity of these models to 
many factors shows the importance of human assessment of a situation 
and the importance of good judgment in assigning utility to forces 
surviving the battle at hand. 
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6. Summary . 

The results of this appendix may be summarized as follows: 

(1) a sequence of one-sided models has been presented which shows 
that the tactics of target selection may be sensitive to 
force strengths, target acquisition process, the type of 
attrition process, and/or the termination conditions of 
combat , 

(2) a sequence of models have been presented which shows some 
preliminary results about the effect of resource constraints 
on firing discipline and concentration of forces, 

(3) tactics for target selection are heavily dependent upon 
"command efficiency," 

(4) concentration of fire always on one target type among many 
occurs as an optimal tactic only when target acquisition 
is not subject to diminishing returns, 

(5) target priorities don T t change over time when one assigns 

a worth to surviving target types in direct proportion to their 
kill rate against you. 
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Appendix C. Target Selection in Lanchester Combat: Heterogeneous 

Forces and Time Dependent Attrition Rates. 

1. Introduction . 

In this appendix we develop solutions to a sequence of models 
for optimal target selection in combat against heterogeneous enemy 
forces. The combat continues over a period of time with a choice 
of tactics available to one side and subject to change with time. 
Optimal target selection rules are developed through the combina- 
tion of Lanchester-type equations for combat attrition and 
deterministic optimal control theory (Pontryagin maximum principle) . 
Both constant attrition-rate coefficients and some special cases of 
variable attrition rates are considered. The synthesis of optimal 
target engagement policies was facilitated by exploiting special 
mathematical structures in these problems. 

Many years ago, Karl von Clausewitz said that if theory 
caused a more critical study of war, then it had achieved its 
purpose. In Appendix B we studied the structure of the optimal 
allocation policies for some basic elementary tactical situations 
described by Lanchester-type equations of warfare. We did this by, 
among other things, contrasting the "best" target selection policies 
for a sequence of scenarios: prescribed-duration battle, terminal- 

control battle, two enemy target types, many enemy target types, etc. 



C-2 



In Appendix B we frequently summarized results and promised the 
details of solution development (for the non-trival cases) to be 
available elsewhere. 

Thus, the purpose of this appendix is to provide supporting 
analysis details for target selection in Lanchester combat against 
heterogeneous enemy force types and/or combat in which the attri- 
tion rates may change over time. All battle scenarios that we 
consider here are for a prescribed duration of time. We also 
present some more general results than those reported in Appendix B: 
target selection for combat against several enemy force types with 
all attrition rates subject to change over time. 

Again, the purpose of this appendix is to provide analysis 
details. The interested reader will find an extensive discussion 
of the structure of optimal allocation policies and model implications 
in Appendix B. Our presentation in this present appendix follows 
the historical order in which we attacked these problems. We 
could have merely reported the solution of the problem in section 4. 
This would have been more concise. We feel, however, that it is more 
valuable to show the reader how we have used insight gained from 
simpler problems to provide guidance for solving more complex ones 
and generalizing results. 

In this appendix we develop the solutions to all problems by 
the mathematical theory of optimal control. For the problems at 
hand we need only make use of the Pontryagin maximum principle [8]. 
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The reader should note that there is a sign difference between 
developments in this country (see p. 108 of [6] or pp. 12-14 of [7]) 
and those in the Soviet Union [1], [8], although both approaches 
yield exactly the same results in applications. 

We wish to impart to the reader through our examination of 
these problems our experience that some ingenuity is required to 
solve applied problems. By taking advantage of the structure to 
some problems, we have found that one can frequently synthesize 
the optimal control to a problem which at first sight appears to 
be too complex to allow this. By the synthesis of the optimal 
control, we mean the explicit determination of the time history of 
the optimal control from initial to terminal time. For the prob- 
lems at hand, this is accomplished by determining a control law by 
the maximum principle and then working backwards from the end of 
the problem by a backwards integration of the adjoint system of 
differential equations for the dual variables which have a boundary 
condition there. 

The organization of this appendix is as follows. First, we 
consider the problem of optimal selection of a target type from among 
two alternatives for some special cases of variable attrition rates. 
Then, we treat the more general case of several target types for 
some special instances of variable attrition rates. Finally, we 
make some observations on the results presented in this appendix. 
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2 . Two Target Types, Some Special Cases of Variable Attrition 
Rates . 

We consider the following prescribed duration battle: 



maximize ry(T)-px (T) - qx„(T) with T specified, 
0(t) 



subject to: 





i X = 

dt 



-0a 1 (t)y, 
-(l-0)a 2 (t)y 
-b 1 (t)x 1 - b 2 



(t)x 2 



> 



x^yX^y £ 0 and 0 < 0 £ 1 , (1) 

where 

p, q and r are values placed on surviving force types, 

X l 5 X 2 anc * y are avera & e f° rce strengths, 
a^(t), a^Ct), b^(t) and ^ 2 ^^ are attr ^ t ^ on rates which 
are allowed to change over time, 

and 0 is the fraction of Y-fire directed at X^ . 

The physical interpretation of this idealized military 
situation is as follows: combat for a known length of time between 

an X force composed of two types of weapon systems (for example, 
infantry riflemen and machine gunners) and a homogeneous Y force 
(for example, riflemen only). The objective of the Y-force 
commander is to maximize the military worth of his surviving forces 
at the end of a battle lasting a known length of time T and to 
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minimize that of the enemy’s* This is accomplished by his choice 
of the fraction of fire, 0, directed at the forces. A 

schematic of this scenario is shown below. It is assumed that a 
military worth can be assigned to survivors of each force-type and 
that this utility is additive. We denote this utility per unit of 
weapon system as p, q and r for the X^, X^ and Y forces, 
respectively. 




Our state equations, i.e., the differential equation con- 
straints, represent the attrition process in this idealized battle. 
We shall refer to attrition as being a "square-law" process when 
the casualty rate is proportional to only the number of enemy 
firers and as being a "linear-law" process when it is proportional 
to the product of the number of enemy firers and remaining targets. 
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For the problems in this paper we assume a "square-law" process. 

The interested reader can find discussions of the physical assump- 
tions which lead to this type of attrition in articles by H. Weiss 
[12] and H. Brackney [5]. The essential points are that new targets 
are acquired at a rate independent of force levels and that fire 
is aimed at point targets. More recently S. Bonder [3], [4] has 
developed formulas for estimating the Lanchester attrition-rate 
coefficient, the rate at which one unit of weapon system destroys 
enemy targets, as a synthesis of the following factors: hit pro- 

babilities, rates of fire, target acquisition rate, weapon system 
projectile- target lethality characteristics, adjustment process. 

In general, the Lanchester attrition-rate coefficient is 
range dependent, and the above formulation (1) takes this into 
account by having the attrition coefficients depend upon time. S. 
Bonder [2] has done the pioneering work on analyzing dynamic combat 
situations with Lanchester-type equations and variable attrition 
rates (also see [9], [10] for more background). For example, consider 
a mobile attack on a static defensive position. The effectiveness 
(Lanchester attrition-rate coefficient) of weapons systems depends 
upon force separation, and this is, in turn, related to time via 
the attack velocity. In [9] we showed how to develop Lanchester- 
type equations with either time or range as the independent variable 
and noted the equivalence of the two formulations (see [10] for 
more extensive results with variable attrition rates). Hence, the 



above was our motivation for studying scenarios with time-dependent 
attrition rates. 

We finally note that when fire is uniformly distributed over 
a target area [12] or the rate of target destruction is constrained 
by the rate of acquisition of new targets and this is inversely 
proportional to target density [5], a "linear-law 11 attrition process 
results. We have elsewhere [11] pointed out that this type of 
attrition structure leads to a fundamentally different structure 
for the optimal target-type selection policies. 

We now develop the solution to the above problem (1) . In 
all cases we consider the special instance when both X-force 
weapon systems are such that 



b i (t> = v 



h(t) and 



b 2 (t) = 



k b 2 h(t) 



( 2 ) 



This physically means that both weapon systems have the same type 
of range capability (for example, quadratic dependence of kill 
rate on range) and the same effective range, although one weapon 
system dominates the other in exactly the same manner at all ranges. 

It seems appropriate to point out that there is a special 
instance when the optimal allocation policy takes a particularly 
simple form. This is when the Y force values surviving X-force 
types in direct proportion to their kill rate against the Y force, 
i.e., p/q = k b 1 / k b 2 = b^O/b^t) = b ± (t=T) /b 2 (t=T) . The optimal 
allocation rule for 0 < t i: T is then 
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C 1 for a 1 (t)b 1 (t) > a 2 (t)t> 2 (t) , 

0*(t) = \ (3) 

^ 0 for a 1 (t)b 1 (t) < a 2 (t)b 2 (t) . 

This was obtained by application of optimal control theory [6] as 
follows . 

The Hamiltonian is given by 

H(t,x i> p i> 0) = -p 1 (t)0a 1 (t)y - p 2 (t) (l-0)a 2 (t)y 

- P 3 (t) {b 1 (t)x 1 + b 2 (t)x 2 }> (4) 

where the dual variables, p^(t) for i = 1,2,3, are the effect 
on the total value of survivors at the end of battle when an optimal 
policy is followed from time t until time T of having an addi- 
tional unit of x^(t) [here we have let y(t) = x^Ct)]. Another 
way to state this is that 

9S 

p i(t) = (t) where S = S (t .x^x^y) is 

i 

the optimal value function. According to the maximum principle, 
the optimal control is determined by 

maximize H(t ,x . ,p . ,0) , (5) 

0 * 0 * 1 1 1 

and this leads to 



r 1 for v(t) > 0, 
0*(t) = i 

** 0 for v(t) < 0 , 



( 6 ) 
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where 



v (t ) = -p 1 (t)a 1 (t) + p 2 (t)a 2 (t) 



(7) 



The adjoint system of differential equations for the dual variables 
is given by 



dp l . 


_ M 


dt 


3x 1 


dp 2 _ 


_ M 


dt 


3x, 


dp 3 _ 


_ iii 


dt 


3x, 



= b (t)p. 



= b (t)p. 



with p 1 (t=T) = -p, 



with P 2 (t=T) = q, 



= 0a 1 (t)p 1 - + (l-0)a 2 (t)p 2 with p 3 (t=T) = r. 



( 8 ) 



By assumption (2) and the adjoint system (8) , it is readily shown 
that 

jh \ 

d »2 ' V 

which, when integrated, leads to 



\ 

P x (t) = -j^ (p 2 Ct) + q> - p. 



(9) 



Using (9), we may write v(t) defined by (7) as follows 



v(t) = 






V 



(-p 2 (t)){k b a^t) - \ a 2 (t)} + ^ q - j-i} a^t) 
1 2 2 b 0 



( 10 ) 
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Assuming that p/q = b^/ b 2> we see that the optimal target selec- 

tion rule as given by (3) is an immediate consequence of (6), (7) 
and (10). We have also made use of the fact that p 2 (t) < which 

is easily shown. 

We now consider the case when 



a- (r) = k h(t) and a (t) = k h(t) . (11) 

1 a l 2 a 2 

This means that all four attrition rates are proportional to the 

same basic time dependence. As we have pointed out elsewhere, both 

the solution to Lanches ter- type equations [9] and the solution to 

the fire distribution problem [11] in this instance are essentially 

the same as for constant coefficients, only with a transformation 

of the time scale. The solution is shown in Table I. 

We develop the solution shown in Table I for the nonrestric- 

tive assumption that k k, > k 'k, as follows. Consider the 

a l °1 a 2 b 2 

quantity v(t)/h(t) which by (7) is seen to be given by 



v(t) 

h(t) 



~ P l (t)k a + P 2 (t)k a ’ 
a l 2 a 2 



Differentiation of (12) leads to 



( 12 ) 



d f v(th 

dt *'h(t)' 



= -h(t)(k 






-k 






>P 



3’ 



(13) 



where we have made use of both the adjoint equations (8) and also 
assumption (2). Observing that h(t) > 0, it is clear that the 
control law (6) is equivalent to 



TABLE I. Solution to Target Selection Problem 
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0*(t) = 



v ( t) 

U for _U.< 0} 



(14) 



Since we develop the solution to this problem by working backwards 
from the end t = T, it is convenient to introduce the "backwards 
time" variable t defined by x = T - t. Observing that 
we have that 



= h(l)(k a 1 k b 1 ‘ k a 2 k b 2 ) P3 (T) 



(15) 



It is easily shown that p^(t) > 0 for all t. Hence, it follows 
from (15) that 



> 0 for a11 T ' 



(16) 



since h(t) > 0 and we have assumed that k k, > k k, . 

a l °1 a 2 °2 

To solve this problem, we work backwards from the end of 
battle at t = T, or equivalently x = 0. At this point we have 



v(t=T) = h(t=T) (pk -qk ). 

a l a 2 



(17) 



Thus, we have two cases to consider. 

In Case (A) : k p ^ k q, we have that v(t=T) ^0 by 

a i a 2 v(t) 

(17) . Consideration of (16) immediately yields that ^ ^ - y > 0 for 

all x > 0 and thus 0*(t) = 1 for 0 £ t < T by (14). 

In Case (B) : k p < k q, we have that v(t=T) <0 by 

a l a 2 

(17) and the assumption that h(t=T) > 0 (the battle ends with 
non-zero attrition rates). Thus 
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with 




Thus 




at some time. Let x^ denote the time when this happens. Then 
for 0 £ x < x^ we have that 0*(x) =0. We can easily determine 



These variable coefficient adjoint equations are readily integrated, 
however, by consideration of (2) and (11), i.e., a 2 (x)/b 2 (x) = 

constant, and the observations elaborated upon in [10], i.e., 
from (18) we can obtain the following linear second order differ- 
ential equation for P 2 ^ x ^ 



P 2 (x) during this interval by considering the adjoint equations 
(8) which may be written as 




= -b 2 (x)p 3 (T) with p 2 (t= 0) = — q , 



and 




= -a 2 (x)p 2 (t ) with p 3 (x=0) = r. 



(18) 




d2p 2 a 2^ 



with 



p 2 (u=0) = -q, 



du 



(u=0) = -r. 
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and 



u 



t> 2 (x)dx • 

0 



Hence, we readily find that 



P 2 ( t ) 



-q cosh 0 (t) 



- r 




sinh 0 (x) , 



where 



(19) 



0(t) = /k 



a 2 S 



h(i)dx . 



Substituting (19) into (10) , we obtain that 



( 20 ) 



'<T> 



“b 



{q cosh 0(x) 



2 *- 



+ r - sinh 0(x)}(k k -k k, ) + k k, q{^- - r— 

k a 2 a l b l a 2 % 2 a l % 2 q 



, ( 21 ) 



where 0 (t) is given above by (20). Observing that is deter- 

mined by the transcendental equation v(t^) =0, we see that we 
have obtained all the results presented in Table I. 
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3 . Several Target Types, Constant Attrition Rates . 

We consider the following prescribed duration battle: 



n 

maximize vy(T) - £ w.x.(T) 

0.(t) i=l 11 

dx. 

subject to: — — = -0.a.y 



with T specified, 
for i = 1 , . . . ,n, 



= “ £ b .x. , 
dt i l 

1=1 



n 



x.,y ^0, 0. ^ 0 for i = l,...,n and £ 0. = 1, 

11 i=l 1 



where v and w_^ for i = l,...,n are values placed on surviving 
specific force types and all other symbols are used in the same 
sense as above. We now present the details behind the solution 
which we stated in a companion paper [11]. 

The Hamiltonian to the above problem is given by 



H(t,x ,p ,0 ) = -y l a p (t)0 - p n+1 l b.x.. (22) 

1=1 1=1 



According to the maximum principle, the optimal control (there is 
only one extremal) is determined by the (trivial) linear program 



maximize H(t,x. ,p. ,0.) 

i ,r i’ 1 

n 



subject to: \ 0. = 1, 

i=l 



0 ± ^ 0 , 
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which in turn leads to 



11 

maximize £ a^(-p^(t) )0^ 



0 i=l 



n 



subject to: £ 0. = 1, 

i=l 1 



0 ^ ^ 0 . 



(23) 



By inspection the solution to (23) is easily seen to be 



V c) - 



(24) 



where <5 is the Kronecker delta and is equal to 1 for i = j 
and zero otherwise and j (t) is the index such that 



a p (t) = minimum (a p a p , . . . ,a p ) . 
] J 1122 nn 



To trace the history of 0L over time, we must consider 
the adjoint system of differential equations given by 

dp i 3H 

- 7 — = - t— = p , ,b . with p . (t=T) = -w. for i = 1, . . . ,n, 

dt 3x. n+1 1 1 l 

l 



and 



dp 



n 



dt 



■ = - — = £ 0.a.p. with p (t=T) = 

3y . L n i ii n+1 

i=l 



v. 



(25) 



Considering (4) , it is easily seen that 



dp . b . 
1 1 



dp b * 
n n 



so that 



p . (t) = T-i {p (t) + W } - W 

i bn n l 

n 



( 26 ) 
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Substituting (26) into (23), we obtain after some manipulation 
that the optimal control is determined by 



n 

maximize £ c.(t)0. 

<t>. i=l 1 1 

n 

subject to: £ 0. = 1, 

i=l 1 



0 ± ^ 0 , 



(27) 



where 



(-P (O) 

c.(t) = r a.b. 

i b ii 

n 



+ “i j 


■ b 

n 






_ (-p„(t))l 


b 


w.j 




for i = 1 , . 


. . ,n. 


i. < 


e. , Y 



(28) 



values enemy survivors in direct proportion to their kill rate 
against his forces, the above problem is equivalent to 



n 

maximize ) 0.a.b. 

, l i i 



0. i=l 

l 



subject to: J 0. = 1, 

i=l 1 



<?> ± > 0 , 



where we have made use of the easily verified fact that p (t) < 0 

J n 

for all time. Thus the optimal control is given by 



0^(t) = 6 for 0 < t £ T, 



(29) 



where j is the index such that 



018 



a.b. = maximum (a- b a b ). 

J J 11 n n 

Hence, for this problem when one values enemy survivors in direct 
proportion to their kill rate against you, the optimal tactic is 
to concentrate all fire on a single target type until it is entirely 
destroyed. 

We now consider the more complex case where one does not 
value enemy survivors in direct proportion to their kill rate 
against you. We shall see that if the battle lasts long enough 
there will be one or more switches in the ranking of target prior- 
ities. As before, since we develop the solution to this problem by 
working backwards from the end t = T, it is convenient to intro- 
duce the "backwards time" variable T defined by t = T = t. At 
the end of battle t = T, we arrange the enemy target types so 
that n is the index such that 

aw = a (-p (t=0) ) = maximum (a_ w. , . . . ,a w ) . (30) 

nnnn 11 nn 

By (23) it is easily seen that 

0*(t=T) = fi . (31) 

i in 

By straightforward continuity arguments, it is readily seen that 

0*(t) = 6. for x € [0,0, (32) 

i m i 

where x^ is the "backwards time" of the first switch in target 
selection. Giving consideration to (32) and observing that 
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= - -j^ 9 we see that for x € [0 ,t-^] we need only consider the 
following equations from the adjoint system (25) 



dp n 



dx ^n^n+1 


with 


dp n+l 


with 


dx 3 n P n 



p n <T ’ 0) “ "V 

P „+1 <T * 0) = v - 



(33) 



and we recall that (re-writing (26)) 
b. 

p.(x) = (p (t) + w } - w. for i = l,...,n. (34) 

l bn n l 

n 

The above initial value problem (33) is routinely solved to yield 

/r 

p (x) = -w cosh /a b x - v/ — sinh /a b x. (35) 
n n n n a n n 

n 

We will now determine what conditions are necessary for a 
change in target selection and the time at which the change occurs, 
To do this we re-write (27) and (28) as 

n 

maximize £ e . (x ) 0 . 

0. i=l 1 1 

1 

n 

subject to: J 0. = 1, 

i=l 1 

0 ± £ 0, (36) 

where 

b. 

e . (t ) = a.w [1 + {(-p (t)) - w }]. (37) 

i li w . b n n 

l n 



We switch at the smallest x for which 
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a .w. 
1 1 



1 + 



b. 

i_ 

w .b 



{ (-P_(t)) - w } 
n n 



= a (-p (t) ) , 

n n 



(38) 



l n _ 

where i = l,...,n-l and certain other conditions (to be deter- 
mined presently) are met. Let k be the index of the target type 
to which fire is first shifted in "backwards time." Observe that 
at t = 0, we have 



a . w. < aw , 
li n n 

for i = l,...,n-l, since the index n has been defined by 

Then for T ± < T < T 2 w ^ ere T 2 t ^ e n ^ ac ^ war ^ s t i- TneM of 

* 

second switch in target selection, we have that 0 .(t) = 6., , 

1 IK. 

thus we must have by (36) and (37) 



(39) 

(30). 

the 

and 



a k w k 



1 + — f— { (-P_(t)) - w } 
w, b m n 

k n 



> a (-p (x)), 
n n 



which may be rearranged to yield 



\ (l kVW * (a k b k- a „ b „ ,( - p „ (T)> 



(40) 



We now show that a necessary condition for fire to be shifted 

from target type n to target type k when one works backwards 

from the end is that a, b, > a b . The proof is as follows. We 

k k n n 

shall show that a, b. £ a b leads to a contradiction. First, we 

k k n n 

consider the special case when a. b. = a b . In this case (40) 

k k n n 

reduces to 



w 

n 




b 

n 





a 



> 



n 
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or 



aw < 
n n 



a kV 



But this is a contradiction to (39) which must hold with i = k. 

In the case when a b > a. b. , then using the fact that (-p (x)) 

nn kk *n 

> w^ for t > 0, we may write (40) as 

(b nV b k u n > , ... 

\rr^rr > <- p „ (T)) > « 

n n k k 

but this leads to a. w. > a w which is a contradiction to (39) 

k k n n 

as before. 

Thus, a. b. > a b and the switch in target selection occurs 
k k n n & 



at 



(b w -b w ) 
k n n k 

*k a, b. - a b 
k k n n 



(-p n ( T=x i)) > ° 



(41) 



so that a second necessary condition is 



b 



k 



b 



> 



w. 



w 



n n 

In other words, we switch to fire at earlier times in the battle 
on the target type which causes attrition proportionally more than 
the ratio of values placed on survivors from the target which 
yields the greatest direct return at the end of battle. 

To recapitulate the above, the target to which fire is first 
shifted (working backwards from the end of battle) has index k 
determined by 
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*k 



= minimum (R.. , . . . , R ) , 

R . >0 

a .b . >a b 
11 n n 



(42) 



where 



R. 

i 



(b . w -b w . ) 
in n i 

a i a .b . - a b 
li n n 



for 



i = 1, • 



,n-l. 



(43) 



The time of switch, 



l 5 



of fire to the 



k th 



target type is 



determined by the transcendental equation 



w 

n 



cosh/a b t.. 
n n 1 




/a b T- 
n n 1 



a k (b kV b nV 

a, b. - a b 
k k n n 



(44) 



This is seen to be a generalization of the case for two target 
types [11]. 

The general pattern of when and to which target types fire 
is shifted as we work backwards from the end of battle does not 
emerge until we have considered the second shift in target selec- 
tion. Since this is dependent upon the evolution of target worth, 
we must further consider the backwards integration of the adjoint 
system of differential equations. From above, we have that 

0*(t) = <S ±k for t € ( V t 2 ), (45) 

where is the ’’backwards time” of the second switch in target 

selection. Giving consideration to (45), we see that for t 6 [t^,t 2 ] 
we need only consider the following equations from the adjoint 
system (25) 
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k 

dT b k P n+l 


with 


iP n+l . 

dx a k P k 


with 



p k (T= V ■ -v 



Vl (T= V " V 



(46) 



where 



\ = R k 



a k (b k W n" b n W k ) 



a k b k 



a b 
n n 



(47) 



V 1 = ^ (W 2 -w 2 ) + V 2 
k b k n 



(48) 



Equation (47) is merely (43) re-written with a chnage in notation. 

Equation (48) is readily deduced when we observe that according to 

(33) a "square law" relates the dual variables p (x) and p in (x) 

n n+1 

for 0 £ t £ t 

a { p 2 (t ) - w 2 } = b {p 2 (t) - v 2 }. (49) 

n n n n n+1 



We further observe that all the dual variables may be expressed in 
terms of p^x) (let n = k in (26)) 
b. 

P ± (t) = ^ (p k (x) + w k ^ ~ for 1 = l>-*-> n - ( 50 ) 

k 



Again, the equations (46) are routinely solved to yield for 
t 6 [t 1 ,t 2 ] 



K 

P k (x) = -W k cosh/a k b k (x-x.^ - — sinh/a k b k (x-x^ 



(51) 



Subsequent arguments are now similar to those given for the 
first switch in tactics. Let j be the index of the target type 
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tovhich fire is shifted secondly in "backwards time." Then, it 

may be shown by similar arguments to above that necessary conditions 

til 

for fire to be shifted to the j target type are that 



a .b . > a. b. > a b , 
] J k k n n 



(52) 



and 



b . w. 

\ V 



(53) 



However, we gain more insight by considering (53) slightly 
differently, for it may be shown that 



b. b, b 

— i > — > — . 

w . w, w 
j k n 



(54) 



It also seems appropriate to consider the military interpretation 
b. 

of the ratio — . We recall that 
w . 
r 

w^ = value per unit of X_^ surviving at t = T, 
b^ = kill rate per unit of against Y. 



Then 



b . 



kill rate per unit of 



X. 



w. value per unit of X. survivors’ 
r i 



Thus, we see that as we progress backwards from the end of battle 
that fire is always shifted to target types with larger ratios of 
kill rate per unit of weapon system per unit value of survivors. 
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By a similar argument as for the first shift in fire using 
(42) and (43) , it may be shown that the target to which fire is 
shifted secondly (working backwards from the end of battle) has 
index j determined by 



S. 

J 



minimum (S 
S.>0 

a . b^ a, b. 
ii k k 

i^k 



1 ’ 



.,s n ), 



(55) 



where 



S. = 



a. (b .w. -b. w . ) 
ilk k l 



i a .b . - a. b. 

ii k k 



for i = 1, . . . ,n. 

i^k 

.th 



(56) 



The time of switch, r^ 9 of fire to the j target type is 
determined by the transcendental equation 

W. cosh/a. b (x -xj + V. y/-^- sinh/a.b. 
k kk21 ka, kk21 

k 



a . (b .w. -b, w. ) 

= _J 2 k k J -, 

a .b . - a, b T 
J J k k 



(57) 



Further shifts in fire follow the pattern established above. 

Based on our above development, we now trace the course of 
battle forward in time. We assume that no force type is ever 
reduced to zero. (If this does happen, though, it may be shown 
that fire is merely shifted to the surviving target type of highest 
priority.) Then, we have that 
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(a) when x^ ^ T 







0.(t) 


= 6. 
in 


for 


0 


£ 


t £ T, 


(b) 


when 


t 2 ^ T > 


T 1 














0*(t) 


" 6 ik 


for 


0 


£ 


t * T - T 1> 






* 

0 ± ( t) 


= 6. 
in 


for 


T 


- 


SC t £ T - t 2 . 


(c) 


when 


T 3 iT > 


T 2 














0*(t) 


= 6. . 
i J 


for 


0 


£ 


t sc T - t 2 . 






0*(t) 


= 6 ik 


for 


T 


- 


t 2 2C t SC T - 






* 

0 ± (t) 


= <S. 

in 


for 


T 


- 


^ c t C 1, 



and so forth. x^ and x^ are determined by (44) and (57), 

respectively, and a similar expression exists for x^. The number 

of switches in target engagement depends upon the scheduled length 

of battle, T. As the battle progresses forward in time, fire is 

b. 

always shifted to a target type for which both a.b. and — are 

i 

smaller than the target type previously engaged. If the battle is 

scheduled to last long enough, then during the initial stages of 

all fire is concentrated on the target type for which both the 

b. 

quantities a.b. and — are larger than any other target type. 

i 
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4 . 



Several Target Types, Some Special Cases of Variable Attrition 
Rates . 

We consider the following prescribed duration battle: 

n 



maximize {vy(T) 
0 i (t) 

subject to: 



- I 

i=l 

dx . 
x 



dt 



dZ = _ 

dt 



w.x. (T)} 
1 1 

-0i a i(t)y 



l b^t) 



with 

for 



T 

i = 



specified, 
1 , • • • , n , 



n 



x.,y ^ 0, 0. ^ 0 for i = l,...,n and £ 0. = 1, 

11 i=i 1 



where all symbols are used in the same sense as above. We shall 
omit some details in the development of the solution to this pro- 
blem, since the development is a direct synthesis of details from 
the two previous problems. In fact, an extensive elaboration on 
the nature of the optimal engagement rules will not be necessary, 
since we shall consider some special cases of variable attrition 
rates for which the solution is similar to the constant attrition- 
rate case considered in section 3. 

In all cases, we assume that the X-force weapon systems 
have performance characteristics sich that 



b^Ct) = k^ h(t) for i = l,...,n. (58) 

i 

As before, we first consider the special case when X-force types 
are valued in direct proportion to their kill rate against the 
Y forces at the end of battle 
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w^ = cb^ (t=T) = ck^ h(t=T) 



(59) 



The optimal allocation rule consequently takes a particularly 
simple form for 0 < t £ T 



0. (t) = 6 . . f v , 

i i,j (t) 



(60) 



where j (t) is the index such that 



a.(t)b.(t) = maximum (a. (t)b_ (t) , . . . , a (t)b (t)). 
3 3 11 n n 



Similar to our previous developments, the above solution is 
developed as follows. The adjoint system of differential equations 
is given by 

dp i 

-7— = b . (t)p with p . (t=T) = -w. for i = 1, . . . ,n, 
dt 1 r n+l r i 1 ’ ’ ’ 



~~ inr " J, ®i a i (t)p i “ ith p n+i (t ‘ T) ■ v - 
i=l 



(61) 



It is readily seen that assumption (58) leads to the result that 



Y 



p i (t) " {p n (t) + V ‘ V 
n 



(62) 



for i = 1 , 
to 



,n. It may be shown that the maximum principle leads 



n 

maximize / 0.c.(t) 

. • i i i 

0. i=l 

n 

subject to: £ 0.-1, 

i=l 1 

* 0 , 



(63) 



C- 29 



where 



c ± (t) 






a i (t)k b 

1 



w. ( 'S 

1 + i ) — il 

( -Pn (t)) K. 



w ^ 






w. 

1 y 



(64) 



The special assumption (59) that force types are valued in propor- 
tion to their kill rate leads to simplification of (64), namely 



c ± (t) 






a (t)k 
x n . 



b n ( t ) 



a i^ b i^ ’ 



n 

whence our result (60), since it is readily shown that p^(t) < 

We now consider the more complex case when enemy force 
types are not valued in proportion to their kill rates. Again, 
we consider a special case when relatively simple analytic results 
are still possible. Thus, we assume that 



a . ( t) = k h (t) for i = l,...,n. (65) 

x a . 

x 

In the case of a mobile attack against a static defensive position, 
assumptions (58) and (65) have the physical interpretation that 
all weapon systems and weapon system combinations have the same 
effective range and the same type of range dependency for their 
kill rate against any target type. As always, we develop the 
solution by working backwards from the end of battle; so it is 
convenient to introduce the "backwards time" t = T - t. Let n 
be the index of the target type fired on at the end of battle. 



Then 
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a 

n 



Then from 



(t= 0) w = a (t=0)(- P (t= 0)) 
n n n 

= maximum (a^(T=0)w^,. 
(63) and (64) for 0 £ t < 



. , a (t=0)w ) . 
n n 

we have that 



( 66 ) 



where is the "backwards time" of the first switch in target 

type at which the Y forces fire. As before, the above "bang- 
bang" optimal control leads to simplification of the adjoint system 
of differential equations, and it readily follows that 



p (t) = -w cosh 0 ( 
r n n n 



/\ 

t) - v/ i — — sinh 9 (t), 
k. n 



(67) 



n 



where 



e n (,) 




( 68 ) 



In deriving the above we have used the same results for variable 
coefficient differential equations used in section 2 and noted 
in [9] . 



Let k be the index of the target type to which fire is 



first shifted in "backwards time." Then for x ^ < x < t ^ we have 
* 

that 0j_( T ) = 5^* To determine the index k, we re-write (63) 



and (64) as 
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n 

maximize £ 0 . e . (x ) 

0 ± i=l 1 1 
n 

subject to: £ 0. = 1, 

i-1 1 



0 . ^ 0 , 

i 



(69) 



where e_^(x) is g iven by 



e i( T ) = a^(t)w^ 



V 



1 + t ~~ I ( — P (t)) - W I 

W i d n n / 



(70) 



The condition that e n (t) > e (x) for x > x- leads to 

k n 1 



k a t (k b "n _k b “k> •= < \ k bJ k a ''b >( - p n <T)) ' 
k k n k k n n 



(71) 



which, in turn, yields that necessary conditions for fire to be 
shifted as we work backwards from the end to the k^ target type 



are 



k k > k k 
a. b, a b 
k k n n 



(72) 



and 



\ ^ 

k n 

> 

w, w 

k n 



(73) 



Thus, it follows that the index k is determined by 



R, = minimum (R- , , 

k R . >0 1 

k k, X >k k 
a . b . a b 
li n n 



’Vi> - 



(74) 



where 
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R. 

1 



WV'b “i> 

ii n 



k k, - k k 
a. d . at) 
ii n n 



(75) 



The ’'backwards time" of switch, x^, of fire to the k ^ target 
type is seen to be given by the transcendental equation 



AT 

w cosh 0 (x.) + v/ - — — sinh 0 (x..) = R, , (76) 

n n 1 k n 1 1c 

a 

n 



where 0^(x) i- s given by (68). This is seen to be a generalization 
of corresponding results in the two previous sections. 

Further details and results are similar to those presented 
in section 3. However, they differ in the same fashion as the 
earlier developments in this section differ from corresponding ones 
in section 3. We finally observe that the optimal target engage- 
ment policies are the same as those given at the very end of 
section 3. Changes in target priority are similarly determined 
(compare (42) and (43) with (74) and (75)). As noted elsewhere 
[9], [11], however, the time scale of battle has been transformed 
as evidenced by a comparison of values for x^ as determined by 
(44) with (76) . 
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5 . Comments . 

In a companion paper [11], the interested reader can find 
an extensive discussion of the structure of the optimal target 
engagement policies for the problems whose solutions have been 
developed in this paper. We have contrasted there the structures 
of the optimal allocation policies for both these problems and 
also other tactical allocation problems. Here, we will make some 
further comments, however, about the optimal target engagement 
policies . 

Let us first note, however, that for the problems considered 

in this paper the optimal control was always "bang -bang," i.e., 

an extreme point of the constrained control variable space. Thus, 

we saw that the optimal tactic was always to concentrate all fire 

on the appropriate target type. It may be shown that there are 

no singular extremals (see pp. 246-247 of [6]) on which the optimal 

control is an interior point of the control variable space. This 

happens, for example in the problem of section 2, because although 

the Hamiltonian is a linear function of the control variable, the 

3H 

coefficient of the control variable, i.e., cannot vanish over 

a finite interval of time. In other words, it is impossible that 



JL(M) = o 
dtW 



Let us now consider the most general case of target selection 
in combat against heterogeneous forces, the problem of section 4. 

We did not include this problem in a companion paper for reasons 
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of brevity. For this problem, our basic assumption was that 

b^(t) = k^ h(t). Then, when survivors were valued in proportion 
i 

to their effectiveness (i.e., kill rate against the Y forces), 
the optimal tactic took a remarkably simple form: always concen- 

trate all fire on the target type for which a^(t)b^(t) is the 
largest. 

To obtain simple analytic results when X-force types were 

not valued in proportion to their kill rate, we further assumed 

that a.(t) = k h(t) . Then as the battle developed forwards in 
i a . 

1 

time, the concentration of all fire on one target type was always 

shifted to a new target type for which both quantities a.(t)b.(t) 
b ± (t) 1 1 

and were smaller than those corresponding to the previously 

i 

engaged target type. 

In this paper we have attempted to show how some seemingly 
complex optimal control problems may readily yield relatively simple 
analytic solutions when special mathematical structures of the pro- 
blem are exploited. We first pointed out such possibilities in a 
recent note [9] (where we inadvertently rediscovered some results 
apparently first observed by R. Isaacs). Although the solution to 
matrix differential equations (such as those encountered in sections 
3 and 4) is, in general, messy at best when specific solutions are 
required, the occurrence of a "bang-bang' 1 optimal control in these 
optimization problems reduced the complexity of the solution appre- 
ciably. We have purposely presented a sequence of problems to 
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show the reader how we set about to exploit such structural pro- 
perties. Our personal opinion is that this could be done more 
frequently in applied areas like operations research where phenomena 
(here military operations) is to be studied. 



c- 36 



6 . Summary . 

In this appendix we have presented the solution details for 
two problems discussed in a companion paper [11] and then consid- 
ered a more general problem. We have shown how optimal controls 
may be synthesized in a simple fashion when special mathematical 
structures of the problems at hand are exploited. By considering 
a sequence of simplified, yet increasingly complex specific pro- 
blems, we have hopefully laid the groundwork for studying more 
general and complex structures in the optimization of combat 
dynamics . 
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Appendix D. Target Selection in Lanchester Combat: Linear-Law 

Attrition Process. 

1 . Introduction » 

In this appendix we develop the solution to a simple problem 
of target selection in Lanchester combat against two enemy force 
types which undergo n linear-law M attrition. In addition to the 
Pontryagin maximum principle, the theory of singular extremals 
is required to solve this problem. Our major contribution is to 
show how to synthesize the optimal target selection policies from 
the basic optimality conditions. This solution synthesis method- 
ology is applicable to more general dynamic (tactical) allocation 
problems. For constant attrition rates we show that whether or 
not changes can occur in target priorities depends solely on how 
survivors are valued and is independent of the type of attrition 
process . 

In Appendix B we have presented some elements of a mathe- 
matical theory of target selection in dynamic combat situations. 

We did this through the examination of the structure of the 
optimal allocation policies for some tactical situations described 
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by Lanchester-type equations of warfare. The purpose of this 
previous appendix was to contrast the structures of the optimal 
allocation policies for various scenarios without cluttering the 
discourse with the mathematical details of solution development. 

In the present appendix we develop results for a prescribed dura- 
tion battle in which enemy target types undergo a "linear-law" 
attrition process (see section 3 below) , We had previously [8] 
just stated these results without justification. 

The problem under study is solved by the mathematical 
theory of optimal control. Its solution, however, requires more 
than the well-known Pontryagin maximum principle [7]: the theory 

of singular extremals (see Chapter 8 of [2]) must be used to solve 
it. A brief discussion of the required theory of singular extremals 
is included in this appendix. By an extremal we mean a battle 
trajectory on which the necessary conditions for optimality are 
almost everywhere in time satisfied. 

The major contribution of this appendix is to show how to 
synthesize the optimal control in combat against target types which 
undergo a "linear-law" attrition process. In this case, singular 
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subarcs (see section 2. below) may be present in the battle trajec- 
tory. By the synthesis of optimal control, we mean the explicit 
determination of the time history of the optimal control from 
initial to terminal time as a function of the initial state of 
the system. There is no general method for the synthesis of optimal 
controls in singular problems [6]; each class of problems possesses 
its own peculiarities. Hence, an understanding of how to synthesize 
the optimal control in this elementary problem is particularly 
important, since it provides insight for more complex extensions 
that we have considered in our subsequent researches. 

The body of this appendix is organized in the following fashion. 
First, we review that part of the theory of singular extremals 
which is required for the solution of the problem under study. 

Next, we present our model and develop the basic necessary condi- 
tions of optimality. Then, we show how to synthesize the solution 
to our problem. This is done for the two cases of import. Finally, 
we make some comments about the structure of the optimal target 
selection policies and extensions. 

2 . The Theory of Singular Extremals . 

In an optimal control problem, the maximum principle may fail 
to determine an optimal trajectory, since the maximization of the 
Hamiltonian may not lead to a well-defined expression for optimal 
control [5], [4] (also see Chapter 8 of [2]). Singular solutions 
usually occur when the Hamiltonian is a linear function of the 
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control variables. However, all problems for which the Hamiltonian 
is a linear function of the control variables do not have singular 
subarcs in their solution. 

The problem that we shall consider has one control variable, 
and it appears linearly in the Hamiltonian. By a singular subarc 
we denote that part of an optimal trajectory on which the maximum 
principle cannot be used to determine the control because the 
coefficient of the control variable in the Hamiltonian is zero (see 
pp. 226-227 of [4]). Then the term "singular soltuion" will be used 
to refer to any optimal trajectory which contains one or more 
singular subarcs. 

To elaborate further, when the Hamiltonian H is a linear 

3H 

function of the control variable 0 , then if 7 ^ = 0 for a finite 

interval of time (or, another way to say this, the coefficient of 

0 vanishes identically for a finite interval of time) , then the 

maximum principle does not determine the control. Observe that in 

this case all feasible values of 0 maximize the Hamiltonian. When 

this happens we determine the singular control by requiring that 

3H 3h 

we remain on the singular subarc, i.e. ^ remains zero. If is 

to be identically equal to zero for a finite interval of time, then 

all its derivatives with respect to time must also be equal to zero. 

We determine the singular control, which keeps the system on the 

singular subarc, by considering as many of the time derivatives of 
3H 

— as are required for the control variable 0 to appear explicitly 
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so that it may be determined from an algebraic equation. Thus, 
in general we consider 



_ _ 3H = _d_.3H. 
U " 30 dt o0^ 



<H) - 



dt 



2 30 



For the problem at hand, the equation 



d 2 r 3H. 



dt 



2 W 



( 1 ) 



= 0 and the canonical 



equations (i.e. both state and adjoint system) lead to an explicit 
expression for the singular control. 

We must further check to make sure that we can get a maximum 
return (in the case when we wish to maximize the criterion functional) 
from use of the candidate singular subarc. The following condition 
(generalized Legendre-Clebsch condition) is necessary for a singular 
subarc to yield a maximum return 

< 2 > 

dt 

It is obtained by examining the negative semidefiniteness of the 
second variation for a special class of explicitly defined control 
variations [6]. For the problem at hand, it suffices to consider 
the generalized Legendre-Clebsch condition with k = 1. Recently, 
Jacobson [3] has discovered a new necessary condition for optimality 
on singular subarcs. This condition is not readily checked, how- 
ever, for the problem at hand, since the details of application are 
extremely messy. 
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3. The Model and Development of Basic Optimality Conditions . 
We consider the following prescribed duration battle 



maximize {ry(T) - px (T) ■ 
0(t) 

subject to: 



qx 2 (T)} 

^1 = 

dt 

dx^ 



dt 



with T specified, 
-( 1-00 a 2 X 2 y ' 



where 



= -fa x - b x 
dt b l X l b 2 2’ 



x^ y x^ 9 y ^ 0 and 0 < 0 £ 1, 



(3) 



p, q and r are values placed on surviving force types, 

x^, x^ and y are average force strengths, 

a^, a^ y and b 0 are constant attrition rates, 

(observe that the a ! s and b ! s are different in nature), 
0 is the fraction of Y-fire directed X . 



In previous papers [8], [9] we have described the basic scenario 
under consideration and also the circumstances which lead to a "linear- 
law" attrition process. As before [8], we refer to attrition as 
being a "linear-law" process when the casualty rate is proportional 
to the product of the number of enemy firers and remaining targets. 

We have also discussed at length the structure of the optimal target 
selection policies and its implications for military tactics 
previously [8], 



We now develop the basic optimality conditions which hold on 



extremals. The Hamiltonian for the above problem is given by 



H(t,x i ,p i ,0) = (-P-L^x-jy + P 2 a 2 x 2 y)0 



+ {"P 2 a 2 x 2 y " P 3 (b 1 x 1 + b 2 x 2 )}, 



(4) 



where p^(t) for i = 1,2,3 are the dual variables corresponding to 
the state variables = y (see [8], [9] for a discussion of 

the military significance of these variables). The maximum principle 
leads to the following non-singular optimal control (there is only 
one extremal) 



The adjoint system of differential equations for the dual variables 
is given by 




Q for P 2 a 2 x 2 < P-^x^ 




(5) 




with p^t = T) = -p, 




with p 2 (t = T) = -q, 




* 0 a l x ;iPl + (1 “ 0) a 2 x 2?2 with P 3 (t = T) = r. (6) 



Conditions (1) for a singular subarc yield for the problem 



at hand 
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Differentiation of (8) and combination with the state equations 
(3) yields the singular control required to remain on a singular 
sub arc 



Checking the generalized Legendre-Clebsch condition for this singular 
subarc, we find after a rather laborious computation which requires 
use of both (3) and (6) that 



since it is readily shown that p^(t) > 0 for all t. Hence, the 
necessary condition is met for the singular path to be optimal. 

In synthesizing the optimal course of battle (backwards from 
the end of the prescribed duration battle) it is convenient to 
introduce 



By (5) and (9) the optimal control may be expressed in terms of 
v(t) as 




v(t ) = -a 1 p 1 x 1 + a 2 p 2 x 2> 



( 10 ) 




for v(t) < 0. 



for v(t) = 0 



for v(t ) > 0 , 



( 11 ) 



We recall that (8) must also hold on a singular subarc. 



Since we develop the solution to this problem by working 
backwards from the end t = T, it is convenient to introduce the 
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"backwards time" variable T defined by T = T - t. Observing that 
-jt = - ^pjr and using both the state equations (3) and the adjoint 
system (6) , we obtain from differentiation of (10) that 



IF" <a lVl ‘ a 2 b 2 x 2 )p 3' 



( 12 ) 



Thus, we see that on a singular subarc on which v(T) = 0 we also 
have that — ~ = 0. Also, it is sometimes convenient to write (10) 



as 



V(T) = - 



p 2 ( t ) 



P 1 ( T ) 

p 2 ( t ") 



-(a^b^x^)- a 2 b 2 X 2 



(13) 



At the end of battle t = T , we have 

v (T = 0) = a 1 px 1 (t = T) - a 2 qx 2 (t = T) . (14) 

Taking (13) into consideration, we see that a point on the singular 
"surface" a^b^x^ = a 2 b 2 x 2 yields a positive, zero, or negative value 
for v(t) at t = 0 depending upon whether is greater than, equal to, 
b l 

or less than — . Hence, by (11) a battle trajectory which has reached 



the singular surface can, in general, only remain on it at the end of 
b l 

battle when = r— . Thus, in synthesizing optimal trajectories we 
q t>2 



must consider three cases* 



b i 

Case (a) £ = — , 

q b 2 

b- 

E- > — 

q V 



Case (b) 



o - 10 



Case (c) 




The solution for Cases (a) and (b) has been described by us in a 
previous paper [8]. 

If we were to plot in Figure 1 the line L f defined by 
a iP Xi = a^qx^, then it would appear above, on, or below the line L 
defined by a^b^x^ = a 2^2 x 2 depending on whether ^ were greater than, 

b i 

equal to, or less than — . This is evident from considering the 



slopes of these two lines. 



rdx„ ' 


a,b. 


'^ X 2 


2 


1 1 




dx n 

^ 1 J 


L a 2 b 2’ 





since, for example, 



£ 

q 



> — implies 

2 



that 



r 

CM 

•3 


"s 


'dx 2 ' 


dx*. j 


T« 


Ldx, J 



The significance of the line L f and its relationship to the 

line L is as follows. The battle is divided into two time phases: 

Phase I for 0£t£t^ = T-X^ and Phase II for T - = t^ <_ t <_ T. 

During Phase I the optimal target engagement policy at a point in 

time is determined by the location of the point on the battle 

trajectory with respect to the line L, which is also the singular 

vt & 

"surfaces.” Above L, 0 (t) = 0; while below L, 0 (t) = 1. When 
a battle trajectory reaches L, it remains on the singular surface 

* a 2 

through use of the singular control 0 = ; . 

a l + a 2 



During Phase II 



Case (a ) 



D-ll 



& 




CM 

X 



Figure I. Optimal allocation for linear-law attrition process- 
Survivors valued in direct proportion to kill rates. 
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the optimal target engagement policy is to use 0 (t) = 1 below 
L*. It may be shown that it is impossible for a battle trajectory 
to cross L 1 during Phase II. 

The above results will be developed in the next two sections 
on the synthesis of optimal control. The following relationships 
readily follow from previously developed results and are required 
to establish the results of the above paragraph 



so that 



where P(t=T) 





r > o 


below L’ , 






v(t=0) > 


{.. 


above L 1 , 




(15) 


* J 1 

0 (t=T) = < 


for 


P (T) below 


L’, 




l 0 


for 


P(T) above 


L’, 


(16) 


(x^ (t=T) ,X£ (t=T) ) . We also note 


that by (12) 




( 


' > 0 


below L, 






dv , . J 
(T)< | 


i ‘° 


on L , 






1 


^ < 0 


above L. 




(17) 



4 . Solution Synthesis When Survivors Valued in Proportion to Kill 
Rates . 



For Case (a) : = -“S 

q b 2 

in Figure 1. Above the line 



optimal battle trajectories are shown 
L with equation a^b^x = a^b^x^ the 
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optimal control is to use 9> (t) = 0 until this line is encoun- 

* 

tered. When a trajectory reaches L, the singular control 0 = 



a l + a 2 



(which keeps the trajectory on L) is used until the end of 



battle at t = T. Below L, 0 (t) =1 is used in a similar fashion. 
To establish these results, we work backwards from each possible 
type of end point of battle. 

At the end of battle t = 0 equation (13) reduces to 



v( =0) = (j^-) [a^b^Ct^) - a 2 b 2 x 2 (t=T)] 



(18) 



since we have assumed ^ . By (18) we see that there are three 

q b 2 

cases to consider depending on the sign of the term in square 
brackets . 



Case (1) 



ai b i x i(t=T) = a 2 b 2 x 2 (t=T) 



This corresponds to when the system ends up on the singular 

* 

subarc. In this case 0 (t=T) = a 2 /(a^+a 2 ), and for 0 £ t £ = 

the ’’backwards time” of the first switch, we use the singular 
* 

control 0 (t) = a 2 / (a^+a 2 ) . Let us note that use of the singular 

control for 0 £ t £ T- results in 4^ =0 so that v(t) = v(t=0) 

I dr 

t 

— dx = 0. At t, = T - t, we switch control, since x.. (t.. ) 

0 dt 11 ’ll 



+ 

= x° or x n ( t- ) = x 



2 1 



This yields three further subcases- 



Subcase (1A) 



a l b l x l ‘ a 2 b 2 X 2 



At t = t^ > 0 we have that a^b^x® = a 2 b 2 x 2 (t^) < a 2^2 X 2 so 
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that we cannot destroy anymore x^. Then we use 0 (t) = 0 for 
^ x ^ T. This is consistent since v(t=t^) = 0 and 

S = P 3 ( a i b l X l _a 2 b 2 X 2 ) < 0 for T ! * T * T - 



dx 2 

(Observe that for t. < x £ T, - — 

1 dr l l 



= a 0 x 0 y so that x 2 (t) > x^x^.) 



This implies that v(x) < 0, and hence 0 (t) = 0 for 0 £ t £ t^ 

= T - v 



Subcase (IB) 



a l b l x l > a 2 b 2 x 2 



A similar argument readily yields that 0 (t) = 1 for 



0 < t £ t^. 



Subcase (1C) 



a l b l x I ■ a 2 b 2 x 2 



We use 0 (t) = a^/ (a^+a^) from the beginning. 



Case (2) a^x^t^) < a 2 b 2 x 2 (t=T) 

Since v(t=0) = (-^-) [a b x - a b x ] < 0, at the end of battle 
X_ 111 2 z z 

•k ^ 

we have 0 (t=T) = 0. Hence, for 0 £ t £ = the "backwards time" 

k 

of the first switch, we use 0 (t) =0. We work backwards from the 
end. Since we are above the line L, = P 3 ( a ^^^ x ^“" a 2^2 X 2^ < 

Hence, v(t) < 0 for all i £ [0,T], and we never do switch. Thus, 

k 

we have that 0 (t) = 0 for 0 i t i T. 



Case (3) a l b l x l^ t=T ^ > a 2 b 2 x 2 (t=T) 

A similar argument to that used for Case (2) readily yields that 



0 (t) = 1 for 0 « t £ T. 
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The above cases are shown in Figure 1. It should be noted that 
the above development depends upon the easily proven fact that p^(t) > 0 
for all t. It should further be noted that, in general, trajectories 
(1A) , (IB) and (1C) will not all terminate in the same point as shown 
in Figure 1, which was drawn this way for simplicity. 

5 ♦ Solution Synthesis when Survivors Not Valued in Proportion to Kill 
Rates . 

b 

We now consider Case (b) : ^ > — . Again, we work backwards from 

q b 2 

each possible type of end point of battle. There are two cases to be 
considered. 



Case (1) : never on singular subarc for finite interval of time. 

Again there are two subcases to consider, depending upon whether 
the system winds up above or below L. 



Subcase (la) a^b^x^(t=T) ^ 3i^b^x^(t-T) 

Since 



we see that 
0 £ T s: T 

0*(t) = 1. 



v(T) 



a iVi 



r p 2] 


(P 1 /P 2 > a 2 b 2 X 2 


i b 2 J 


(b L /b 2 ) a i h i\ 



* 



> 



v(t=0) > 0 and hence by (11) 0 (=T) = 1. Hence for 

= the ,f backwards time" of the first switch, we use 
We work backwards from the end using this control. Since 



d v 
dx 



P 3^ a l b l X l~ a 2 b 2 X 2^ > ° 
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when we are below L and we stay there by using 0 (t) = 1, we have 
that v(t) > 0 for all t € [0,T], and hence we never switch. Thus, 
0*(t) = 1 for 0 £ t i. T. 

Subcase (lb) a l b l X l^ t=T ^ < a 2 b 2 X 2^ t=T ) 

Again there are two further subcases to consider, depending upon 
whether the system winds up above or below L'. 

Subcase (lbl) a^b^x^(t=T) < a 2 b 2 X 2^ t= ^ and 

a i px i ( t=T ) < a 2 q x 2 (t = x) 

* 

In this case we wind up above L 1 and hence by (16) 0 (t=T) = 0, 

Since we are above L, — - < 0 for all t by (17). Combining this 

with (15), it readily follows that v(t) < 0 for all x € [0,T]. 

* 

Thus, 0 (t) = 0 for 0 £ t £ T. 

Subcase (lbll) a^b^x^(t=T) < a 2 b 2 X 2^ t= ^ and 

a lP x l(t= T ) > a 2 qx 2 (t=T) 

In this case we wind up below L' at the end. By (15) and (16), 

* 

we have that v(t=0) > 0 and 0 (x=0) = 1. We work backwards from the 

end. Since we are above L, <0 by (17) while we remain above 

L. Thus v(t) decreases as t increases. There are two further 

subcases depending upon whether v(t) decreases to zero before the 

line L is encountered. Let x^ be such that v(x^) =0. If L 

has not been reached at x^, then v(x) for x > x^ is negative and 
* 

0 (x) = 0 for x^ £ x £ T. It is also possible to just reach L 
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when v(t^) = 0- 

singular subarc) 

then — > 0. 
dx 



In this case (assuming that we don ! t remain on the 
v(t) > 0 for t > since we pass below L and 



Case (2) : on singular subarc for finite interval of time 

Considering (15) and (17), it is readily seen that this can only 

happen when a^b^x^(t=T) < a 2k2 x 2^ t=T ) and a^px^(t=T) > a 2 qx 2 (t=T). 

As usual, we work backwards from the end of battle. By previous 

& 

arguments it is readily seen that we use 0 (t) = 1 for 0 £ t £ t^, 

and at t = we must have = a 2^2 X 2^ T l^’ We USe t * 16 

singular control 0 (t) = a^/ '(a^^^) ^ or £ t £ There are 

three further subcases. 



(1) 


x 1 (t 2 ) 




> x 2 (t 2 ) 


< X 


(2) 


x 1 (t 2 ) 




, x 2 (t 2 ) 


= x! 


(3) 


x 1 (t 2 ) 


■ 


> x 2 (t 2 ) 


= x! 



We omit the trivial discussion of these cases. 

Thus we see from the above that there are six possible cases for 
the history of combatant force strengths in this prescribed duration 
battle: 

(1) started below L and never reached L, 

(2) always above L f , 

(3) started above L ! and end up above L but below L* 
without ever reaching L , 
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(4) 


end up above L but started below L 
on L for finite interval of time, 


and did not remain 


(5) 


started above 
of time, 


(or on) L and 


were on 


L for finite interval 


(6) 


started below 
time . 


L and were on 


L for 


finite interval of 



These six cases are shown in Figure 2. Case (c) : 




is 



similar to Case (b) . 



6. Comments . 

Elsewhere [8] we have contrasted the structure of the optimal 
target engagement policies in Lanchester combat when the engaged tar- 
get types undergo a "linear-law" attrition process with that for other 
tactical scenarios. An important question to be answered in such studies 
is whether target priorities change over time. We have discovered that 
for the scenarios which we have so far studied the answer to this 
question is determined solely by whether or not surviving target 
types are valued in direct proportion to their kill-rate capabilities. 

For the case of constant attrition rates, changes in target priorities 
over time can only occur when survivors are valued in excess of their 
kill-rate capabilities. This is true when the engaged target types 
are undergoing either a "linear-law" attrition process or a "square- 
law” one (see [9] for a discussion of the "square-law" case). 

We now discuss how the above principle applies to the problem 
at hand. When a linear utility is assigned to enemy survivors at the 
end of battle in direct proportion to their kill-rate (effectiveness) 



Case (b) 
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CNJ 

X 



Figure 2. Optimal allocation for linear -law attrition process - 
Survivors not valued in direct proportion to kill rates. 
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against friendly forces, then the optimal target selection policy 
depends only upon the location of the battle trajectory with respect 
to the singular "surface" L (see Figure 1). Thus, target priorities 
don't change over time (they can become equal, however). When one 
target type is assigned utility in excess of its effectiveness (i.e., 
p/q > b^/b^), then at time t^ there will be a switch from tactics 
being determined by the location of the battle trajectory with respect 
to the singular "surface" L to being determined by location with 
respect to the line L f (see Figure 2), It may be shown that t^ 
depends on the particular battle trajectory under consideration and 
no trajectory can "penetrate" L f . 

The methodology for solution synthesis developed in this paper 
is applicable to more complex tactical situations of greater military 
significance. Our work here lays the foundations for the study of 
the optimal allocation of supporting weapon systems (e.g., artillery, 
tactical air support, etc.) against "area targets" (e.g., troop 
concentrations). Typical questions of interest to be answered are, 
"Considering several infantry companies individually engaging enemy 
units of like size, what is the 'best' utilization of supporting 
artillery fires?" or, "What is the 'best' utilization of Naval fire 
support in amphibious assaults?" 

In a previous paper [8], we have pointed out that the structure 
of the optimal allocation policies in Lanchester combat is basically 
determined by whether there are constant attrition returns over time 
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per unit of weapon system employed or diminishing returns. In the 
present paper we have studied target selection with diminishing 
returns over time, i.e., "linear-law 11 attrition process. It should 
be noted that there is a problem in the literature with similar solu- 
tion structures, the continuous version of Bellman’s stochastic gold- 
mining process (see pp. 222-233 of [1]). When there are diminishing 
returns over time from the use of a device subject to breakdown, then 
the problem of maximizing the return from use of one device in either 
of two potential locations has a similar structure to the optimization 
problem in Lanchester combat studied here. The interested reader 
should compare the solution as shown in our Figure 1 with that of 
Theorem 1 on p . 231 of [1] and also our Figure 2 with Figure 4 on 
p. 232 of [1]. When the stochastic gold-mining problem is re-examined 
by modern optimal control theory, new insights are gained into the 
operation of maximizing the return from a resource subject to break- 
down or loss, and we shall discuss this in the future. 
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Appendix E. The Theory of State Variable Inequality Constraints. 

1. Introduction . 

Optimal control problems involving inequality constraints 
on a function of the state variables with no explicit dependence 
on the control variables have been treated only in recent years. 

The pioneering work of the Russian R. Gamkrelidze in 1959 (see 
Chapter VI in [22]) was followed in the U.S.A. by that of L. Berko- 
vitz [3], S. Dreyfus [12], and Bryson, Denham, and Dreyfus [9]. 
Berkovitz and Dreyfus [5] subsequently have shown the equivalence 
of the results of Gamkrelidze and Berkovitz with those of Dreyfus. 
McIntyre and Paiewonsky [20] have written an excellent survey ar- 
ticle on the theory of state variable inequality constraints (SVIC) 
and summarize theoretical results through 1966. 

Furthermore, in 1967 McIntyre and Paiewonsky [20] remarked 
that "the optimal control problem with state space constraints 
does not appear to be well understood." Our own experience bears 
this out. We have recently pointed out [27] that certain theoret- 
ical results cited in the literature are inadequate by themselves 
to determine optimal trajectories in a certain non-autonomous 
class of problems but must be supplemented by a result due to 
Gambrelidze [22]. 

Even more incredible is the fact that for two-sided varia- 
tional problems (i.e. two-person zero-sum deterministic differen- 
tial games) we are not aware of any adequate treatment of inequality 
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constraints of functions of the state variables only (the control 
variables not explicitly appearing). Although R. Isaacs does con- 
sider such a problem (the "War of Attrition and Attack" [16]) , we 
will show that his solution is unsupported by proper analysis, and 
we will indicate a similar problem for which his approach fails to 
yield optimal strategies. L. Berkovitz does not consider problems 
with state variable inequality constraints (SVIC) in [4] ; and al- 
though A. Friedman claims to treat such problems [13] (see also 
Chapter 6 in [14]), he fails to develop the appropriate necessary 
conditions of optimality which must hold when such a constraint is 
active (i.e. on a constrained subarc). 

In this appendix we shall review some necessary conditions 
of optimality which must hold on a constrained subarc in a one- 
sided problem with a SVIC. Following the approach of L. Berkovitz 
[4], we may readily extend these conditions to differential games. 
We propose this to ONR as a future research task. 

2 . Basic Approaches . 

There are two basic approaches to treating a SVIC in an 
optimal control problem. The first and more direct approach con- 
sists in adjoining the state-variable constraint directly to the 
criterion functional with an additional Lagrange multiplier. This 
approach has been considered by Chang [10] and Speyer and Bryson 
[23] (see also [18]). We will not consider this method of direct 
adjoining further in this appendix. 
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The second method consists in adjoining the time derivative 
of the state-variable constraint to the criterion functional with 
an additional Lagrange multiplier. We say [9] that the problem 



the first time (total) derivative of the state-variable constraint 
which explicitly contains the control variable(s) is the k^ . 

This is the approach originally considered by Gamkrelidze [22] for 
a first order SVIC and further extended to higher order SVIC by 
Bryson et al. [9]. Furthermore, this is the approach that we shall 
take in this appendix. 

3 . Necessary Conditions of Optimality When Time Derivative of 
Constraint Is Adjoined to Return Functional . 

It suffices to consider a problem with a single control 
variable and a single inequality constraint on the state variables. 
Without loss of generality we may assume that the problem 1 s plan- 
ning horizon is for a fixed period of time. We further consider 
a (possibly different) inequality constraint on the state variables 
at the end of the planning horizon. 



has a k order state variable inequality constraint (SVIC) when 



Let us therefore consider the problem 



T 



maximize 



u(t) 




with T specified, 
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dx. 

subject to: for i = l,...,n, (1) 

u(t) e U (u(t) is scalar control variable 

suitably restricted) , 

C(t,x.) £ 0 (scalar inequality constraint on 

state variables) , 

ij;(x.(T)) ^ 0 (scalar inequality constraint on 
state variables at terminal 
time) , 

where we assume that all functions are smooth enough to insure the 
existence of all partial derivatives required in the following 
analysis. We shall consider separately the cases of a first order 
SVIC and a k th (k > 1) order SVIC. 

a. First Order SVIC . 

Adjoining the first time derivative of the state-variable 
constraint to the return functional, we have that the Hamiltonian 
is given by [8], [9], [11], [22] 



n 



H(t,x i ,p i ,u) = L(t ,x i ,u) + l p i f i (t,x i ,u) - p(t) — , (2) 

i=l 



where 



y(t) 



= 0 


for 


c(t jX^ 


< 0 


:> 0 


for 


CCt.x^ 


= 0 
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We may, of course, write (2) in expanded form as 

n 

H(t,x i ,p i ,u) = L(t ,x^ ,u) + l p 1 f i (t,x 1 ,u) 

i=l 

-M ( t){|^+ l ^ . (3) 

i=l l 

The adjoint equations for the dual variables are given by 



where p_^(t) the dual variable corresponding to the state var- 

iable x^. Let us assume (as is invariably) that all state variables 
are specified at t = 0, i.e. we are given x^(t=0) = x^ for 
i = l,...,n. Hence, p i (t=0) is unspecified, and we have the fol- 
lowing boundary conditions for the dual variables at t = T 




dp^ 

dt 



3H 

3x. 

l 



3T n 3f -s a 

i 3=1 i i 



p.(t=T) 






for i = l,...,n where 



= 0 for ^(x^CT)) < 0 , 

^ 0 for i|;(x^(T)) = 0 . 



(5) 



Conditions (5) were apparently first observed by Funk and Gilbert 
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[15]. (A special case of conditions (5) was earlier observed by 
Arrow and Kurz [2] although their derivation is only heuristic.) 

Its proof readily follows along the lines of L. Neustadt f s develop- 
ment in {21]. (It should be noted that (5) has been stated for 
the prescribed duration problem. It need not hold in other cases 
(for example, in the case when the planning horizon ends when a 
target set (exclusive of time) is reached).) 

On a constrained subarc on which C(t,x^(t)) = 0 for 
0 £ t^ £ t £ £ T we have that 

^7 (t,x ± (t)) = 0 ' for t 1 < t < t 2 , 



and hence the control is determined by 




J, !e £ i (t ’V u " ) ■ 0 

x=l i 



( 6 ) 



The Lagrange multiplier p(t) is determined by 




!^+ I 



i=l 



3 f i 

p i 3u~ 



- p(t) 




(7) 



Bryson, Denham, and Dreyfus [9] derived that on a constrained sub- 
arc (of finite length in time) a necessary condition of optimality 
is 



p(t) £ 0 



for £ t £ t 2 . 



( 8 ) 
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For this problem (1) Gamkrelidze (see Chapter VI in [22]) (who did 
not observe (8)) developed the following additional necessary con- 
dition 

y(t) « £ 0 for t 1 £ t £ t 2 . (9) 

We have recently pointed out [27] that many workers [1], [8], [9], 
[11] have overlooked the importance of (9). It should be noted 
that results corresponding to (8) and (9) have not been developed 
for differential games (see, in particular, [4], [14], [16]). 

Furthermore, there are corner conditions (for a discussion 
of corner conditions within the framework of the classical calculus 
of variations see p. 38 of [6] or p. 367 and p. 571 of [7]) that 
must be satisfied upon entering to and exiting from a constrained 
subarc. (These important conditions are never mentioned in [1] or 
[2].) Let t^ denote the entry time to the constrained subarc and 




Figure 1. Entry to and Exit from Constrained Subarc. 
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Furthermore, let t^ denote a left-hand limit, i.e. t^ = lim t. 

t " t i 

t ^ t i 

It has been established [9] that the total discontinuity in the 
adjoint variables can be taken at the entry corner (i.e. t = t^) . 
Thus , we have aX entrance corner 



, -x / +x / +x 3 C 

fi (t i > ■ Pi (t i > - ^ c i > 377 



( t l’ X i.) f ° r ± = *» n > ( 10 ) 



and 



_ _L _L 

H(t x ) = H(tp + y ( t ^ ) (t 1 ,x i ) . (11) 

A similar result was first given by McIntyre and Paiewonsky [20]. 
However, there is a sign error in their equation (52). Furthermore, 
the dual variables can be taken to be continuous at an exit corner. 
Thus , we have that at_ exit corner 

P i^2 ^ = P i ( ‘ t 2 + ^ for i = > (12) 

and 

H(t 2 ") = H(t 2 + ) . (13) 

It is readily shown that (12) and (13) imply that 



y(t 2 ) = 0 . 



(14) 
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b . Higher Order SVIC . 

In the case of a k tk (k>l) order SVIC, the first time 
derivative of the state-variable constraint which explicitly con- 
tains the control variable is the k tk . Thus, on a constrained 
subarc on which C(t,x^(t)) = 0 for O^t^^t^t^^T the 
control is determined by 



d k c 

dt k 



= 0 



for 



tl < t < t 2 , 



and we further have 



d n C 

dt n 



0 



for t^ < t < t 2 , 
for n = 1 , . . . ,k - 1 . 



The Hamiltonian is then given by [8] , [9] 



(15) 



(16) 



n 



d k C 



H(t,x ,p ,u) = L(t,x ,u) + £ p f (t,x ,u) - y (t) — t - , (17) 

1 X X # ~a -L -L -L i , 

1=1 dt 



where 



p(t) 



= 0 for C(t,x^) < 0 , 



^ 0 for C(t,x i ) = 0 , 



and the adjoint equations for the dual variables are given by 



dPi 

dt 



3H 

9x. 



9x. 



n 3f . 



. - 1, p j + u(t) 

i j=l i J 



a / d k c\ 

9 *i \dt k /’ 



( 18 ) 
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for i = 1, Again, we assume that initial conditions are 
given for all the state variables (i.e. x^(t=0) = x° for 

i = l,...,n). Then the boundary conditions at t = T for the 
dual variables are again given by (5) . 

On a constrained subarc, the control is determined by (15). 
As derived by Bryson, Denham, and Dreyfus [9], a necessary condi- 
tion of optimality on a constrained subarc is 



y (t) ^ 0 for t 1 t ^ t 2 , 



(19) 



where the Lagrange multiplier (t) is determined by 



!£- o =^ + 

3u du 



r 9f i , . 8 /d k C \ 



( 20 ) 



It remains to discuss the corner conditions (see [9], [20]). 
At an entrance corner we have that 



and 



, -x , +, , dC , N , v 3 ( d n c\ 

>i (t i > ■ p i (t i > + v o aTT (t i ) + J "n 9^: fej 

x n=l i N dt ' 

, + N 3 /d k_1 c\ 

- u(t i > 1 " 

I ^ t l ^ H( ' t l ^ _ V 0 3t ^1 “ V n 3t ( dt _n j 

( + \ 3 / d k_1 C \ 

1 3 t (dt k -V’ 



( 21 ) 



( 22 ) 



where for j = 0,1,..., k-2 are undetermined constants 

(multipliers) . (Again, our results (21) and (22) differ in sign 
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with the corresponding results given by McIntyre and Paiewonsky 
[20].) Finally, at an exit corner we simply have 

P i^ fc 2 ^ = P i^ t 2 + ^ for 1 = 1 »***» n » (23) 

and 

H(t 2 ‘) = H(t 2 + ) , (24) 

which imply that 

y(t 2 ") = 0 . (25) 

4 . Application of Theory to Tactical Allocation Problems . 

In this section we apply the theory discussed above to two 
tactical allocation problems. Both these problems have first order 
state variable inequality constraints. The first problem (tactical 
air-war campaign) is a one-sided version of R. Isaacs’s "War of 
Attrition and Attack {16]." However, we shall consider versions 
for which previous analysis [16] does not lead to an optimal policy. 
The second problem is the fire programming problem studied by 
Isbell and Marlow [17]. (See also [25], in this report as Appendix 
A, where results are obtained in a heuristic fashion (the determi- 
nation of boundary conditions for the dual variables following [17]) 
without the appropriate theory of SVIC.) We shall consider several 
versions of both these problems. We will not develop complete solu- 
tions (including synthesis of optimal policies as a function of 
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initial conditions) to these problems but concentrate on developing 
the basic necessary conditions of optimality. 

a. Tactical Air-War Campaign . 

We consider a one-sided version of R. Isaacs's "War of Attri- 
tion and Attack [16]." We consider the case when one of the com- 
batant forces always flys ground support missions. 

(1) Original Version of Isaacs . 

We consider the problem facing the X-force commander 



T 

maximize f { x ( 1— <{> ) - y) dt with T specified, 
+ (t) 0 



subject to: 



dx 

dt 



nu 



9 



= 

dt 






* C 2 X > 



x ,y ^ 0 and 0 ^ <J> ^ 1 , 



(26) 



where 

x and y are the number of X and Y aircraft, respectively, 

c^ is the rate at which one X aircraft shoots down Y air- 
craft (Lanchester attrition-rate coefficient) , 

m^ and m^ are replacement rates, 

<f> is the fraction of total X aircraft which fly counter-air 
missions (and hence shoot down aircraft) . 
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Clearly, the constraint x ^ 0 can never be binding. The Hamil- 
tonian is given by 

H(t ,x^ ,p^ ,<j>) = x( 1— 4 .) - y + 4- p 2 (m 2 -<j)C 2 x) 

+ y (t) (m 2 -<()C 2 x) (27) 

where 

{ = 0 for y > 0 , 

2; 0 for y = 0 , 

and p^ is the dual variable corresponding to x. The adjoint 
system of differential equations for the dual variables is 



dt 

dp 2 

dt~ = 

with boundary conditions 



— ( 1— 4> ) + <f>c 2 (p 2 +y) , 



1 > 



at t = T 



Pl (t=T) = 0 , 



since x(T) > 0 (assuming T > 0) and 



where 



P 2 (t=T) = v 2 , 




for y(T) > 0 , 
for y(T) = 0 . 



(28) 



(29) 



(30) 



( 31 ) 
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When y > 0, the control law is determined by the maximum 
principle. This leads to 




for 



for 



(~P 2 (t)) 

(-P 2 (t)) 




(32) 



The possibility of a singular solution (see Appendix D for a further 

dp 2 

discussion is excluded by the fact that since -j-^— ^ 0, it is im- 



d / 3H \ DH 

possible to have dt*\^") = ® when — = 0. Hence, the extremal 
control (it is also optimal, since we shall see that there is only 
one extremal) is well-defined by the control law (32) almost every- 
where in time (i.e. the extremal control is determined by (32) 
except at a finite number of isolated instants in time when 



<-p 2 (t)) 




Furthermore, the locus of points for which (-p 9 (t)) = — 

l c 2 

* 

(and <J> (t) consequently changes from 1 to 0 as an extremal 
'’penetrates" this surface) may be referred to as a "switching" or 
"transition" surface. The discontinuity in the control at such a 
locus implies that the trajectory in the state space has a comer 
(without any path constraint), i.e. a discontinuous change in the 
slope of the state variable trajectory. There is no tangent to 
the path at the switching time, although both the left-hand and 
right-hand derivatives do exist. It is well-known (see, for example, 
p. 125 of [8] and also references to the classical calculus of 
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variations in Section 3. above) that the following conditions must 
hold at such a corner 

P i (t ) = p i (t + ) for i = 1 > • • • >n , 

H(t") = H(t + ) , 

where t is the time just before the corner (left-hand limit). 

It should be noted that these vital corner conditions are never 
mentioned in [1], [2], [16], or {24]. However, in all these refer- 
ences they are always implicitly assumed. As we will presently 
see below, the corner conditions may take a different form at the 
entrance to a constrained subarc (i.e. problem with SVIC when 
Gamkrelidze 1 s approach (see pp. 265-267 of [22]) is used), and this 
implicit assumption is then no longer valid. 

On a constrained subarc on which y(t) = 0 for t^ £ t £ t^ 
the control is determined by ^ = 0 and hence 

* (t) * £or h “ c ' c 2 • (33) 

3H 

The multiplier y(t) is determined by the condition = 0 and 
hence 

y (t) = — — - p 9 (t) . (34) 

C 2 

The condition that y(t) ^ 0 yields that on a constrained subarc 



we must have 
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(-Po(t)) ^ 



(35) 



Differentiating (34) and combining with (29), we find that 



dPo 

y(t ) = - -I = _! < o , 



(36) 



so that Gamkrelidze’s condition is always satisfied on a constrained 
sub arc . 

Denoting the time of an entrance corner by t^ and that of 
an exit corner by t^ , the corner conditions (10), (11), (12), and 
(13) yield that 



p ? (t 1 ) = - 



(37) 



/ “K 

p 2 ( c 1 ) = 



" y(t i +) ’ 

c 2 



(38) 



and 






(39) 



When there is an exit from a constrained subarc at t = t^, then 
(using (14) and (36)) it is readily shown that (38) becomes 

- - 7~ 2 - ‘V'P • <40) 

Having developed the basic necessary conditions of optimality, 
let us now consider the synthesis of the optimal control. By the 
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synthesis of optimal control, we mean the explicit determination 
of the time history of the optimal control from initial to terminal 
time as a function of the initial state of the system. In synthe- 
sizing an optimal trajectory there are two cases to be considered: 

Case (a) y(T) = 0 , 

Case (b) y(T) > 0 . 

We now show that an optimal trajectory cannot end at t = T with 
y(T) = 0. 

For Case (a): y(T) = 0, there are two further subcases, 

corresponding to whether or not the trajectory was on a constrained 
subarc for a finite period of time immediately before the end at 
t = T. Thus, we consider 

Subcase (i) y(T) = 0 with y(t) = 0 for t^ ^ t ^ T 

where t^ < T . 

In this case 

P? (t=T) = - -r- - p (t=T) , (4i) 

2 

where 



P ( t=T) ^ 0 , 

since we are on a constrained subarc. However, it is readily shown 
that this is incompatible with (31) which may be re-stated as 
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P 2 (t=T) = v 2 (42) 

where 

v 2 ^ 0 . 

Hence, Subcase (i) is impossible by the inconsistency of (41) and 
(42) , both of which are necessary conditions of optimality. 

Subcase (ii) y(T) = 0 with y(t) > 0 for T - 6 ^ t < T 
where 6 > 0 . 

Since y(T) = 0 with y(t) > 0 for T - 6 £ t < T where 
6 > 0, we must have (considering the state equations (26)) 

<J>*(t) =1 for T - 6 <: t < T . (43) 

By (31) (or (42)) and (29), we have 

P 2 (t) = v 2 - (T-t) (44) 

where 

^ 2 ^ 0 . 

Thus, we can find e > 0 such that for T - e £ t < T, we have 

P,(t) > ~r , (45) 

2 

and hence considering the control law (32) , we are led to a contra- 
diction of (43). Thus, Subcase (ii) is similarly impossible. Hence, 
optimal trajectories are only possible in Case (b) . It had not been 
shown previously [16] that Case (a) is not consistent with an optimal 
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policy. (In fact, the possibility of Case (a) is not even considered 
in [16], [24].) 

For Case (b) : y(T) >0, we have by (31) that 

P 2 (t=T) = 0 . (46) 

Using this terminal condition, a backwards integration of the adjoint 
equation (29) yields 



(~P 2 (t)) = t , 



(47) 



where, for convenience, we have introduced the backwards time x 
defined by t = T-t. Hence, in the case when y(t) > 0 for all 
t, it is easily seen that 




for x > — , 

C 2 

for T < ± , 



(48) 



where we have considered (32) and (47). Thus, in this special case 

(when y(t) > 0 for 0 ^ t ^ T — ) the solution shown in Table I 

c 2 

readily follows. Combining the solution shown in Table I with a 
forward integration of the state equations (26), it is easy to show 
that this solution applies under the following conditions: 



either (a) m^d- — ) < m 2 - , 



(49) 



or (b) 



m l C 2 (T " 7 ? * m 2 ‘ C 2 X 0 ’ 

c^m« _ _ 

y o > T 2 (T " + ( c 2 x o- m 2 )(T - 7 ? 



( 50 ) 



2 
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Table I. Optimal Policy in 

Tactical Air-War Campaign 
When y(t) > 0 for all t. 



Case (1) : T £ 



<J> (t) = 0 for 0 g t £ T . 

(Always fly ground support.) 



Case (2) : T > 



* 

<P (t) = 



. 1 ' 



for 0 g t £ T - 



for T t £ T . 

C 2 
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It remains to develop the solution in Case (b) when neither 
(49) nor (50) holds. Thus when 



C 2 m l 1 1 

y 0 £ ~ 2 ~ (T- — ) + (c 2 x 0 -m 2 )(T- f-) , (51) 

we have y(t.) = 0 where t 1 < T — . In this case y(t) > 0 

2 i 

for 0 £ t < t. and recalling (37) that p„(t, ) = , it is 

1 2 1 c„ 

* z 

readily seen that $ (t) = 1 for 0 ^ t < t^. Furthermore, 

<J>*(t) = — — for t. £ t £ T — . (52) 

r C, X 1 c„ 



We observe that by (40) 



P 2 ( t 






(53) 



Clearly (35) is satisfied. Thus, both (8) and (9) are satisfied so 
that it is optimal "in the small" to remain on the constrained sub- 
arc. Finally, it is readily shown that (39) and (46) are consistent. 
The final solution (the extremal trajectory is optimal, since it is 
unique) may be expressed in a particularly simple form and is shown 
in Table II. 



(2) A Time-Dependent Attrition-Rate Coefficient . 

If the approach used by Isaacs I 16 ] is applied to the above 
problem, then it is assumed that it is optimal to drive y(t) to 
zero and keep it there for t ^ T — whenever possible. However, 
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Table II. Optimal Policy in 

Tactical Air-War Campaign 



Case (1) : T ^ 



■<J> (t) = 0 for 0 <; t <: T . 

(Always fly ground support.) 



Case (2) : 



T 



> 



C 2 



For 



0 ^ t £ T — 

C 2 



* 

4> (t) = 



1 

m2 




t £ T , 



when y > 0 , 
when y = 0 . 

<P*( t) = 0 . 



For T 
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this need not be true and, indeed, is not true in general as the 
following example shows. We consider the case when the attrition- 
rate coefficient c^ is a function of time. 

T 

maximize J (x(l-<t>) - y) dt with T specified , 

+ (t) 0 

, . dx 

subject to: — = m^ , 

" 4> c 2 (t) x , 

x,y ^ 0 and 0 £ <f> £ 1 . (54) 



We will consider only the development of the basic necessary 
conditions of optimality. The analysis of the preceeding section 
applied through (35). However, c 2 is now a function of time. 
Re-writing (34) as 



p(t) 



1 

c 2 (t) 



" » 



(55) 



we have 



y(t) 





- l , 



(56) 



so that Gamkrelidze’s condition (9) is only satisfied on a con- 
strained sub arc when 



dc 



2 



£ [c 2 (t)] 2 . 



dt 



(57) 
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The corner conditions (10) through (13) now take the form 



p 2 (t l _) = ' 



C 2 (t l) ’ 



(58) 



/ "K 

p 2 <t 1 > - 



vv - •■o > 



(59) 



and 



P2^ t 2 ^ P2^ t 2 ^ 



c 2 (t 2 ) 



(60) 



Considering (14) and (56) , it is easily seen that 



y U) 



=-/: 2 fcv£-K 



and hence for ^ t ^ 



y(t) = t 0 - t H 7 — v 7 — r* 

2 c 2 (t 2 ) C 2 (t) 



(61) 



Considering (61) , we see that when there is an exit from a con- 
strained subarc at t = t^, we have 



P 2 (t l +) ' 



c 2 (t 2 ) - ‘V'i' 



(62) 
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(3) Linearly Increasing Attrition-Rate Coefficient . 

It seems appropriate to consider a special case of a time- 
dependent attrition-rate coefficient in order to more clearly illus- 
trate that there are restrictions on when it is optimal to keep 
y(t) equal to zero for a finite interval of time. Let us therefore 
consider the special case when c 2 ^ t ^ a li near function of time, 

i.e. 

c^Ct) = kt . (63) 

In this case, (57) holds (this implies that Gamkrelidze 1 s necessary 
condition (9) is satisfied on a constrained subarc) only for 




(64) 



Furthermore, (58) now becomes 



p 2 (t i > = 



k h ’ 



(65) 



so that by using (29) we find that for 0 ^ t ^ t^ 



P 2 U) ' ‘ 'kt, ■ - ‘1 



+ t - t. 



( 66 ) 



Considering the control law (32), the above expression (66) yields 



that 
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. 1 


l" 


for 


0 * c < 4 - 




♦ (t) =< 


1. 


for 




(67) 



Let us denote the time of switch in tactics by t so that we have 

s 



t 

s 




( 68 ) 



Integration of the state equations (54) using (67) then yields 

for t £ t £ t- 
s 1 

x 1c m Ic 

y(t) = y Q + m 2 t — (t 2 -t 2 ) 1— (t 3 -t 3 ) . (69) 

The unknown quantity t^ can then be determined by solving the 
equation 



y(t x ) - 0 , (70) 

where y(t) is given by (69) and t is given by (68). We have 
not worked out further details. 

Furthermore, if we are never on a constrained subarc then 



* I 1 

* (O - < 


for 


t - c > a 




(° 


for 




(71) 
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We observe that 



and 



4 (t=0) = 0 , 



4) (t=T) = 0 . 



If there is a switch in tactics, then (ignoring the pathological 

case) there must be two, which we denote as t and t . These 

S 1 S 2 

are the roots of the equation 



T - t = 



s kt 



(72) 



Consequently, for T £ — we have 

A 



4 (t) = 0 for 0 £ t £ T 



(73) 



Also, for T > — we have 

A 



0 for 0 £ t < t 



where 



4 (t) - <1 for t < t < t , 

S 1 S 2 
.0 for t < t <: T , 

s 2 



*. -1-VdTT. 



and 






(74) 



(75) 



( 76 ) 
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It seems appropriate to briefly discuss what this partial 
analysis has yielded. First of all, we (the X-forces) don’t begin 
the campaign by shooting down Y aircraft but wait until our ef- 
fectiveness rises enough over time. Intuitively we may think of 
this as manifesting that we should not shoot down planes today 
(initially) because we can do so much better tomorrow. Secondly, 
we adjust our tactics so that (64) holds when we keep the number 
of Y aircraft at the zero level. We note that we don’t always 
drive y(t) to zero whenever we can, but we wait so that (64) can 
hold. We have not worked out the conditions on the initial force 
levels under which a constrained subarc occurs. 

(4) Time Variations in Integrand of Payoff . 

Let us further consider an example in which the payoff from 
allocating aircraft to ground support missions can change over time. 
In other words, we try to take account of changing effectiveness of 
the supporting weapon system (here aircraft). The model is 

T 

maximize f {a(t)x(l-<j)) - b(t)y} dt with T specified , 

♦ (t) 0 

, . dx 

subiect to: ~r- = m- , 

J dt 1 

dy 

dt = m 2 " ^ C 2 X ’ 

x,y ^ 0 and 0 ^ <J> ^ 1 , (77) 

where C 2 is again assumed to be constant. 
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Again, we shall focus on the development of basic necessary 
conditions of optimality and an interpretation of the parameters 
on which the optimal policy depends • The Hamiltonian is given by 



H(t,x i ,p i ,<f.) = a(t)x(l-<f>) 


- b(t)y + P^m^ + p 2 (m 2 ~(f)C 2 x) 




+ u(t)( 


m 2 -<jic 2 x) , 


(78) 


where 






( = 0 


for y > 0 , 




u(t) < 








for y = 0 , 





and the adjoint equations for the dual variables are 



dp i 

= - a(t) ( 1— 4> ) + <t>c 2 (p 2 +y) » (79) 

dp« 

^ = b(t) , (80) 

with boundary conditions at t = T again given by (30) and (31) . 

When y > 0, the control law is determined by the maximum 

for (-p,(t)) > pp- , 
z c 2 

for (-p 9 (t)) < pp- . (81) 

2 



principle. This leads to 



<t>*(t) = 
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The probability of a singular solution, however, is not as easily 
excluded as before. Although it is readily shown to be possible 
d /3H \ 3 H 

to have — I— ) = 0 when — = 0 , complete details as to whether 

dt \d<j> J d<p 

there can be a singular subarc haven’t been worked out at this 
time . 

Again, on a constrained subarc on which y(t) =0 for 

t^ £ t £ t 2 the control to keep y(t) =0 is given by (33). The 

3H 

multiplier p(t) is determined by the condition — = 0 and 

o<p 

hence 

y(t) = - p (t) . (82) 

2 

The condition that p(t) ^ 0 yields that on a constrained subarc 
we must have 

(-p_(t))i^- (83) 

2 

Differentiating (82) and combining with (80) , we find that 

y(t) = - — 77 - b(t) , (84) 

c 2 dt 

so that Gamkrelidze's condition (9) is only satisfied on a con- 
strained subarc when 



b (t) £ 



1 da 
c 2 dt * 



(85) 
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The corner conditions (10) through (13) now yield 



a ( t.) 

P 2 (t i ) = ~ . • (86) 

+ a(t l> + 

P 2 (t l } = " ~ 2 p(t l > * (87) 



and 



+ a(t 2 ) 

p 2 (t 2 ) - p 2 (t 2 ) — 



( 88 ) 



Considering (14) and (84), it is easily seen that for t^ <> t ^ t^ 



1 

y(t) = — {a(t 7 ) - a( t) } + / b(s) ds , 



(89) 



and thus when there is an exit from a constrained subarc at t = t^ , 
we have 



p 2 ( ' t l ^ c / b ( s ) ds * 



(90) 



From the above analysis we see that the "weight" of the 
ground support missions, i.e. a(t) and b(t), are determining 
factors in ascertaining optimal allocation tactics. From this we 
conclude that another type of model (one that considers the conse- 
quence of allocating aircraft to ground support missions) may be 
more appropriate. In particular, the model (26) (due to A. Mengel) 
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ignores the dynamics of ground combat (i.e. the interaction of the 
supporting X and Y weapon systems with the battle of ground 
forces) . We propose this to ONR as a future research task. 

b . The Isbell-Marlow Fire Programming Problem . 

We consider a problem about the optimal distribution of fire 
over enemy target types. This problem was first studied by Isbell 
and Marlow [17] (the same analysis is apparently given in [24]). 
However, we have shown that the solution given in [17] is not valid 
for all ranges of model parameters [25] (reproduced in this report 
as Appendix A) . (There are some gaps in our solution development 
that we rectify in Appendix F, although our previous solution [25] 
is not modified.) Additionally, we developed in [25] a purely 
algebraic method for constructing a solution to such problems, as 
opposed to the geometric method (which failed to uncover a completely 
correct solution) employed in [17]. 

However, in [25] we still used the same heuristic method 
introduced by Isbell and Marlow [17] to determine boundary condi- 
tions for the dual variables. There are some other technical gaps 
having to do with the treatment of the non-negativity of the force 
levels (state variable inequality constraints) . We re-consider 
the development of certain key necessary conditions of optimality 
within the framework of the theory of SVIC . This is the first 
such application of the theory of SVIC to such problems to the 
best of our knowledge. 
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(1) Original Version . 
We consider the problem 



maximize {ry(T) - px^T) - qx„(T)} with T unspecified , 
+(t) 1 2 



subject to: 



dx^ 

dt 



= - 4>a y , 



dx 2 

dT = ~ (1 “*> a 2 y ’ 



m _ 
dt 



Vl ' b 2 x 2 • 



x i ,x 2 ,y ^ 0 and o ^ ^ ^ i , 



where 

x^Ct), x^Ct), y(t) are force levels, 
p, q, r are utilities assigned survivors, 
a l’ a 2’ ^i» ^2 are ( constant ) attrition-rate coefficients, 
and <j> is the fraction of Y fire directed at X^. 

The battle terminates at t « T with either (a) x^T) = x^(T) = 0 
or (b) y(T) = 0 (i.e. it is a "fight to the finish"). As in [25] 

we shall denote the five terminal states of battle corresponding to 
the above two conditions as for i = 1,...,5 (see [25] for 

precise definitions) . 

We will focus on the development of necessary conditions of 
optimality. The synthesis of optimal trajectories from these is 
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carried out in Appendix A (see also Appendix F) . The Hamiltonian 
is given by 



H(t,x i ,p i ,«ji) = -p^a^y - p 2 (!-<()) a 2 y - p^b^+b^) 



- p 1 (t)(f)a 1 y - p 2 (t) (l-4> )a 2 y 



(92) 



where 



y^t) 



= 0 for x. > 0 , 

l 9 



^ 0 for x. = 0 . 

l 



The adjoint system of differential equations for the dual variables 
is 



dp^ 

dt 



dp 2 

dtT 



dp 3 

dt~ 



b 2 p 3 ’ 

^ a l(Pl +y l> + ( 1- ^) a 2^P2 +y 2^ ' 



(93) 



(94) 



(95) 



Deferring the discussion of boundary conditions for the dual var- 
iables until later, we note the transversality condition 

H(t=T,x.,p*,4)*) = 0 . (96) 

When x *l > x 2 > the control law is determined by the max- 

imum principle. This leads to 
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* 

* (t) = 



! 



0 for a 1 (-p 1 (t)) < a 2 (-p 2 (t)) . 



1 for ^(“^(t)) > a 2 (-p 2 (t)) , 



(97) 



As we showed in [25] , it is easily seen that it is impossible to 
d /3H \ 

ha ye — j = 0 when — =0 so that it is impossible to have 
a singular solution. 

Without loss of generality, we may consider a constrained 

subarc on which x^(t) = 0 (and > 0) for t^ <> t <> t ^ The 

* 

control is clearly <J> (t) = 0. The multiplier y^(t) is determined 

3H 

by the condition — = 0 and hence 



The condition that y^(t) ^ 0 yeilds that on a constrained subarc 
we must have 



Differentiating (98) and combining with (93) and (94) , we find that 



so that Gamkrelidze 1 s condition (9) is only satisfied on a constrained 




(98) 



a^-p^t)) 2: a 2 (-p 2 (t)) . 



(99) 




( 100 ) 



subarc with = 0 when 



a l b l 1 a 2 b 2 ’ 



( 101 ) 
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since it is readily shown that p^(t) ^ 0* The corner conditions 
(10) and (11) at an entrance corner yield that 



) = P 1 ( t 1 + ) + p^t]*) , 


(102) 


P 2 ( t l ) = P 2 ^ t l + ^ > 


(103) 


— -j- 




p 3 (t l ^ = p 3 (t l ' ’ 


(104) 



and 



p l (t l > ‘ ^ p 2 (t l > ' 



(105) 



Let us now consider the boundary conditions on the dual var- 
iables at t = T. We use the same notation for the terminal states 
of battle as used in {25] (reproduced in this report as Appendix A) . 
Without loss of generality, we can assume that a ^b^ > a 2 D 2 * T ^ en 
Gamkrelidze * s condition (101) implies that no extremals can lead to 
, and it may be shown that there is no inconsistency with having 
extremals lead to . Terminal states , C^, and C<- are treated 
in a common fashion, since for all of them the time of the end of 
battles is determined by y(T) = 0. Hence, y(T) =0 is a specified 
equality condition on a state variable at an unspecified terminal 
time. Considering this fact (see p. 75 of [8]) and (5), we have 
for states C , , and C<- 
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where 



where 



P 1 (t=T) 


= -P 


+ V 1 




(106) 


( = ° 


for 


X 1 > 


0 , 




( * 0 


for 


X 

h- 1 

II 


0 , 




P 2 (t=T) 


= -q 


+ V 2 




(107) 


(- 


for 


X 2 


0 , 




f 2s 0 


for 


X 2 


0 , 





P 3 (t=T) = v 3 (108) 

where v 3 is unrestricted. 

Furthermore, the transversality condition (96) yields in all these 
cases (i.e. for , and C^) 

P 3 (t=T) = 0 . (109) 

In particular for : x^(T) > 0 > x 3 (T) = y(T) = 0, it 

is readily seen by (101) that this terminal state cannot be reached 
by being on a constrained subarc with = 0 for a finite interval , 

of time. Thus, can only be reached when x 3 (T) = ® but 

x (t) > 0 for t < T. The boundary conditions (106) and (107) on 
the dual variables now take the form 
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Pl (t=T) = ~p , (110) 

and 

P 2 (t=T) = -q + V 2 , (111) 

where ^ 0 . 

The latter condition (111) was conjectured to hold in [25], but we 

could not at that time justify it. Since x^(T) = 0 but X£(t) > 0 

* 

for t < T, we must have <(> (T) = 0, and hence by (97) we have 

a l P 

(-p(t=T) ) > -±- . (112) 

a 2 

Then, combination of (111) and (112) yields that 

a p 

— < (-p (t=T)) <: q , (113) 

a 2 Z 

which was a result used in [25] that we could not justify (due to 
our, at that time, lack of knowledge of the theory of SVIC) . 

(2) A Case of Time -Dependent Attrition-Rate Coefficients . 

As we discussed in [27], for problems with autonomous dynamics 
(8) may well imply that Gamkrelidze's condition (9) is satisfied. In 
non-autonomous cases this need not be the case and highlights the 
importance of Gamkrelidze T s condition. This work here complements 
our results presented in [26] (reproduced in this report as Appendix C) , 
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where we omitted considerations pertaining to the non-negativity of 
the force levels. Let us therefore consider the problem 



maximize {ry(T) - px-. (T) 
4>(t) 



subject to: 



dx^ 

dt 



dx r 

dt 



= 

dt 



x 1 > x 2 »y ^ 0 



- qx 2 (T)} with T specified 

— 4>a 1 ( t)y + r^(t) , 

-(l-i}))a 2 (t)y + r 2 (t) , 

-b 1 (t)x 1 - b 1 (t)x 2 + s ( t) , 
and 0 £ <J> £ 1 , 



(114) 



where r^(t) , r^(t) , s(t) are replacement rates. Again, we focus 
on the development of some necessary conditions of optimality. 

The Hamiltonian is given by 



H(t,x i ,p i ,<j>) = P 1 (t) ( — d>a 1 (t) + r^(t) } + 

P 2 ( t) { ~(l _( l ) )a2^ t )y + r 2 ^ t ^ + P 3 (t)(-b 1 (t)x 1 - b 2 (t)x 2 + s(t)} 

+ y. (t)(-<|>a ,(t)y + r..(t)} + y„(t){-(l-i|>)a 9 (t)y + r (t)} , 

ill/ ^ Z ( 115 ) 



where 



y i (t) 



= 0 for x. > 0 , 

l 

:> 0 for x. = 0 . 

l 
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Equations (93) through (97) still hold with a^, , b^, and b 2 

now being functions of time. 

On a constrained subarc on which x.. (t) = 0 (and x > 0) 

dx x 

for t, £ t £ t„ the control is determined by - — =0 so that 
1 Z at 

* t-^(t) 

* Ct) = 1^1)7 for h * c • (116) 

3H 

The multiplier u^Ct) is determined by the condition — ■ = 0 and 
i o(p 

hence 

Pl(t) = - j t y {a 2 (t)p 2 (t) - a 1 (t)p 1 (t)} . (117) 

The condition that |j^(t) ^ 0 yields that on a constrained subarc 
we must have 



a^(t) (-p^(t) ) i: a 2 (t) (-p 2 (t) ) . (118) 

Similar to what we did in [26] , let us now consider the special case 

when a.(t) = k g(t) for i = 1,2. Then (117) becomes 
i a . 



y 1 (t) 




(k 



P 2 




Pi> 



(119) 



Differentiating (119) with respect to time and combining with (93) 
and (94) , we find that 



y L (t) 



Po(t) 

77 CO < a 2 (t)b 2 (t) ■ a !(t)bi(t)} , 



( 120 ) 
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a 1 (t)b 1 (t) :> a 2 (t)b 2 (t) , (121) 

since again p^(t) ^ 0. The corner conditions (102) through (105) 
still hold only with a^ and a 2 being functions of time. 

(3) Several Target Types . 

In [26] (reproduced in this report as Appendix C) we first 
considered a scenario with one weapon system in battle against n 
target types. Our previous treatment ignored the non-negativity 
restrictions on the force levels. To rectify this gap we consider 



maximize (vy(T) - £ w^x^(T)} with T specified 



<f>,(t) 



i=l 



dx . 

subject to: — — = -6 .a .y for i = 1 , . . . ,n , 

J dt i* 7 



, n 

P-. - T b.x. , 

dt , L , l l 



i=l 



x^,...,x n> y ^ 0 , <j)^ i 0 for i = , 



and ) d) . = 1 
i=l 1 



( 122 ) 



We focus on the development of necessary conditions of opti- 
mality. To avoid being encumbered by too many symbols, we will con- 
sider the case when the - force is driven to zero and assume 

that x 0 , • • • , x >0. In this case, however, we must consider a 
2 n 
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slightly different form for the Hamiltonian. (See pp. 108-109 in 

[8] for a discussion of the difference between Hamiltonians denoted 

* 

there as H and H ; the Hamiltonian given by (123) is H in the 
notation of [8]. As pointed out in [8], whether one uses a Hamil- 
tonian of the form (92) or (123) is a matter of taste.) 



where 



and 



n n 

H(t, x .,p. 4l ) - y{ l a i <-P i >^ p - P n+1 l b x 

i=l i=l 



n n 

+ A (1 - l <J> i ) + y(-<() 1 a 1 y) + £ 

i=l i=l 



y(t) 



= o 


for 


> 0 , 


:> 0 


for 


= 0 , 



Y i 



= 0 


for 


♦i 


> o , 


:> 0 


for 


+i 


= 0 . 



(123) 



The adjoint system of differential equations for the dual variables 



is 



dp i 

= b i P n+1 for i = 1 , . . . ,n , (124) 

and 

dp n+l n 

~dt <f> 1 a 1 (p 1 +y 1 ) + l <t> i a i p i . (125) 

i=2 
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On a constrained subarc on which x..(t) = 0 (and x„ , . . . ,x 

1 Z n 

* 

> 0) for ^ t ^ , part of the control is clearly (j>^ (t) = 0. 

To determine the multiplier y(t), we must take a slightly differ- 
ent approach than previously used above. This development will show 
the reader the origin of the form of the Hamiltonian (123). 

Following Gamkrelidze (see pp. 265-267 of [22]), on a con- 
strained subarc the maximum principle takes the following form 



where 



maximize h(t ,x . ,p . ,<J> . ) 

♦i 



n 

subject to: £ <f> . = 1 , 

i=l 1 



+ . s> 0 
i 



for i = 1 , . . . ,n , 



dx.. 

zi Vi y 2 0 • 



(126) 



n a 

.(t,x ,P ,* ± ) = y{ l a i (-p.)* 1 } - P n+1 l b.x. 

i=l i=l 



(127) 



We re-write (126) in a more convenient form to apply the well-known 
Kuhn-Tucker conditions [19]. (Since the constraints are linear, it 
is well-known that the Kuhn-Tucker constraint qualification [19] is 
satisfied. See [2] for a further discussion.) 
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maximize h( t ,x . ,p . ,<(>.) 

+ i 

n 

subject to: £ <J> . = 1 , 

i=i 1 

-<f>^ <; 0 for i = l,.. M n , 

<^l a l y ^ 0 , (128) 

Then the Lagrangian for the optimization problem (128) on the con- 
strained subarc (which turns out to be the Hamiltonian (123)) is 
given by 

n 

LC4» i »P * Y i » V i> = h(t,x 1 ,p i ,(j) i ) + X(l- l 4^) 

i=l 

n 

+ p ( -<4> 1 a 1 y) + l Y ± 4> i (129) 

i=l 

= H(t,x i ,p i ,(j) i ) . 

Applying the well-known Kuhn-Tucker conditions [19] to (128), we 
have that necessary and sufficient conditions for the maximum to 
the concave program (128) are 



9L_ 

94,. 



|i- - -P^y - X - + r x 

-p.a.y - X + y. =0 for i = 

K i i l 

* 

y(4> 1 a L y) = 0 , 



= 0 

2 ,. 






(130) 

(131) 



(132) 
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= 0 for i = l,...,n , (133) 

and 

y>Y i ^ o. (134) 

Let us assume that there is an isolated maximum to (128) 

(thereby ignoring the troublesome case of alternate optima, which 

merely presents inessential complexities; it is readily shown that 

(128) cannot have alternate optima except at isolated points in time) . 

* 

Then we let i = J be the index such that <f>j =1. By the comple- 
mentary slackness relationship (133) , we have = 0 so that (131) 
yields 

X = (-p J )a J y. (135) 

Using (135) we may re-write the n equations (130) and (131) as 

y a ]/ “ Y i = y ^ a j p j -a l p l^ ’ ( 136 ) 

and 

Y i = y{a J (-p J (t)-a i (-p i (t))} for i = 2,...,n. (137) 

Assuming that y > 0 (otherwise there is no allocation problem) , the 
Kuhn-Tucker optimality condition Y^ ^ 0 for i = l,...,n yields 

a J (-p J (t)) £ a i (-p i (t)) for i = 2,...,n. (138) 

In fact, for the maximum to be isolated, we must have 

aj(-pj(t)) > a^(-p^(t)) for i = 2,...,n and i t J. (139) 
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We are therefore left with one equation for the two remaining 
undertermined multipliers y and 



y a^y - Y x = y(a jPj - a lPl ) . (140) 

Let us note that once we reach the constrained subarc and x^(t=t^) = 0, 
x^(t) remains equal to zero for £ t £ t^ = T , since the model 
(122) does not allow for replacements. Thus by Gamkrelidze 1 s device 
the constraint 4>^a^y ^ 0 is active for t^ < t £ T. Observing that 
the constraint 4>^a^y ^ 0 being active guarantees that the constraint 
4^ ^ 0 is satisfied (we note that a^ > 0 and y > 0) , we set 
= 0 to obtain 



u<t) * 57 <a j p j- a i p i ) • 



(141) 



Then the Kuhn-Tucker optimality condition y k 0 implies that 



a l ( — p i ( ^ a^-p^t)), 



(142) 



on the constrained subarc. Differentiating (141) and combining the 
result with the adjoint equations (124) , we obtain 



y (t) = 



Vi (t) 



(a J b J- a l b l } ’ 



(143) 



so that Gamkrelidze ! s condition (9) is only satisfied on a constrained 
subarc with x^ = 0 when 



a i b i k a jV (144) 

since it is readily shown that p . . (t) > 0 for 0 £ t < T. 

n+1 
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Thus, we have proved the following theorem: 



Theorem E.l : For the problem (122) , an optimal 

policy has the property that if we annihilate 
, then we must have 



a i b i * 



a j b j 



where J is the index of any target type upon 
which fire is later concentrated. 



Although the above does not complete the analysis of this 
problem, we end our discussion here. We propose to ONR that the 
complete discussion of the conditions under which it is optimal to 
drive a force level to zero be a future research task. 
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Appendix F. The Isbell-Marlow Problem Revisited. 



1. Introduction . 

We have included in this report as Appendix A some of our 
previous work [9] on the Isbell-Marlow fire programming problem [6] 

(the same results are apparently given in [7]). This work [9] revises 
some even earlier work (see Appendix A of [8]) that we had done on 
this problem. Since the writing of [9] in June 1971, we have uncovered 
a gap in our development of the synthesis of the optimal control (how- 
ever, the solution presented in [9] remains valid). (By the synthesis 
of optimal control, we mean using the basic necessary conditions of 
optimality to explicitly determine the time history of the optimal 
control from initial to terminal time as a function of the initial 
state of the system.) Thus, it is the purpose of this appendix to 
revise our earlier work [9] (repreduced in this report as Appendix A). 
The results we obtain from this further examination of the Isbell- 
Marlow problem are of considerable interest, since they provide an 
understanding of solution phenomena (multiple extremals and a dominated 
payoff) that we have encountered in a differential game [10]. 

In [9] we showed that extremals were not unique for a certain 
range of model parameters and tried to develop necessary and suffi- 
cient conditions for optimal paths to lead to all the terminal states 
of combat. (By an extremal we mean a battle trajectory on which the 
necessary conditions of optimality are almost everywhere satisfied in 
time.) In the pioneering 1956 work of Isbell and Marlow [6] (the 
Pontryagin maximum principle was only announced in 1956 [1], although 
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Professor Magnus Hestenes of the USA had apparently first given the 
now standard control formulation (as well as developed optimality 
conditions) and urged others to follow his approach in 1949 [3]) one 
can find several gaps (boundary conditions for dual variables, treat- 
ment of state variable inequality constraints (SVIC)) in the light 
of subsequent control theory developments. In [9] we followed Isbell 
and Marlow* s heuristic treatment of the determination of boundary 
conditions for the dual variables and did not consider the theory of 
SVIC. We rectify these gaps here. 

Furthermore, we feel that one of our major contributions in 
[9] was the development of a purely algebraic method for the deter- 
mination of the optimal control, as opposed to the geometric method 
employed in [6] . The reader should note the difficulty of applying 
their geometric approach to a similar problem with a state space of 
dimension greater than three. We feel that this is essentially 
impossible, whereas our algebraic method (like algebraic geometry in 
n-dimensional space) appears to be readily applicable to such problems. 

2 . Statement of the Isbell-Marlow Fire Programming Problem . 

For the reader 1 s convenience we re-state the problem originally 



studied by Isbell and Marlow [6] 
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maximize {ry(T) - px (T) - qx„(T)} with T unspecified, 
♦<t) 1 2 

dx l 

subject to: = - <J>a y, 

dX 2 

d~t (1 "^ )a 2 y ’ 



& = -bx - bx 

dt D 1 X 1 D 2 X 2’ 



( 1 ) 



x l ,x 2 ,y ^ 0 and 0 ^ ^ 1> 



where all symbols are defined in the next section. 

The battle terminates upon reaching the terminal states defined 
by (1) x^(T) = ^(T) ” 0 and (2) y(T) = 0. Upon further analysis, 

it has been convenient to consider that there are the following five 
"target sets" for this problem: 



C 1 : X]L (T) = 0, x 2 (T) > 0, y(T) = 0, 

C 2 : X]L (T) = 0 before x 2 (T) = 0, y(T) > 0, 

C 3 : X]L (T) = 0 after x 2 (T) = 0, y(T) > 0, 

C 4 : x x (T) > 0, x 2 (T) = 0, y(T) = 0, 

C 5 : x x (T) > 0, x 2 (T) > 0, y(T) = 0. 

The reader should note that in the above problem statement T is 
referred to as being undetermined. This is because T is determined 
by entry to one of the above five target sets. This is a function of 
the control applied, and hence before an allocation rule is given, 
it is unspecified. 
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3 . Notation . 

The symbols which are used in this appendix are defined as 
follows : 

A = A(R,z) = [z 2 (R-1) - R]/(z-l) 2 , 

B = B(R,z) = A(z-l) 2 /z 2 = [z 2 (R-1) - R]/z 2 , 

a l ,a 2 , ^l , ^2 = constant a ttrition-rate coefficients, 

C. for i = 1,2, 3, 4, 5 = the i— part of the terminal surface 

as defined in section 2, 

D(C.) = domain of controllability for C_^, 

g(P°,R,z) = term in equation of the locus of points for 
which P^ = P^, 

h(P°,R,z) = term in equation for boundary surface between the 

regions from which optimal paths lead to and , 

p,q,r = utilities assigned to surviving X^, and Y 

forces respectively , 

p.(t) for i = 1,2,3 = dual variable corresponding to 

x ± (t) (x 3 (t) = y (t) ) , 

P. for i = 1,2, 3, 4, 5 = payoff associated with an extremal 

leading to C^, 

P = (x^jX^jy^) = point m the initial state space, 

Q = (-p 2 (t=T))/q 

R = a l b l /(a 2 b 2 ) » 
s = s(x 1 ,x 2 ) = b 1 x 1 + b^x^ , 

t^ = time at which X^ is annihilated, i.e. x^(t^) = 

t^ = first time at which 2b 1 x 1 (t 2 )x° + b 2 (x°) 2 = a^y 2 (t^) for an 
extremal leading to , 

T = total time for the battle, 



v = v ( t ) = a 2 P 2 (x) - a 1 p 1 (x), 



F-5 



w 



a (b p (t=T)+b p) 

= cosh /a 0 b 0 t (C. ) = 7 — -r- 7 — — - , 

22 l v 4' P 2 (t=T) (a 1 b 1 -a 2 b 2 ) ’ 



x l> x 2»y = average force strengths; with initial values x ^, x 2> 

y 0> 

. r-TT- > “l (b l'l- b 2 P> R - 6 

2 ' cosh ¥2 t 1 <C 5 ‘ T (a 1 b 1 -a ; b 2 ) - F^T- 

6 = a x p/ (a 2 q) , 

4> = fraction of Y-fire directed at X^, 

t = n backwards time” from the end of the battle defined by 

= T - t, i.e. the time remaining before the end of the battle, 

T-. (C . ) = "backwards time" of the first switch in tactics for 
1 1 

extremals leading to C^. 



Additionally: 

S th 

Cj_ refers to C<_ (the 5— part of the terminal surface) which 
is reached by extremals with a switch in tactics, 

S S 

= payoff associated with an extremal leading to . 



4 • Summary of Solution . 

There are four cases to be considered 



(1) 


6^1, 


(2) 


R - /R(R-l) < 6 < 1, 


(3) 


6 = R - /R(R-l) , 


(4) 


0 £ 6 < R - /R(R-l) , 




For Case (1) : 6^1, 



in Table I of Appendix A. Our re-esamination here changes no details 



of the analysis presented there. 
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The results of our additional analysis (which includes use of 
the theory of SVIC) are presented in Tables I through V. Supporting 
details will be presented in subsequent sections only when they differ 
from our previous analysis (see Appendix A) . 

For Case (2) : R - /r(R~IT < 6 < 1, the solution is shown in 

Table I. Although this solution presented in Table I (Appendix F) is 
identical to that presented in Table II of Appendix A, there is a 
subtle difference in how it was obtained: our newer analysis (which 
invokes some new results from the theory of SVIC about the boundary 
conditions for the dual variables) has revealed that the domains of 
controllability do not "overlap" so that the extremals are unique. 
Hence, the extremal control turns out to be the optimal control, since 
it may be shown that the problem does indeed possess a solution. 
Previously (see Appendix A) , we thought that the domain of control- 
ability for C^, denoted by D(C^), had an intersection of positive 
area (to be more precise, positive measure) with other domains of 
controllability . 

It seems appropriate at this time to point out to the reader 
that the optimal control has been expressed as an open-loop control. 
For an open-loop control u = u(t;xQ,tg) one specifies the control 
as a function of time t during the length of the planning horizon 
0 ^ t ^ T. Such a control only depends on the parameter t and the 
initial conditions x^,tQ. This control is not directly influenced 



by the current state of the system. On the other hand, one can 
consider a closed-loop (or feedback) control u = k(x,t) which depends 
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Table I. Solution to Isbell-Marlow Problem for 
R - /R(R-l) < 6 < 1. 

Nonrestrictive Assumption: R > 1, i.e. a^b^ > a 2 b 2 

Case (2) : R - /R(R-l) < 6 < 1 where <5 = a^p/ (a 2 q) 



Terminal State 

( X l^ t l^ = 0 

Cj. J x 2 (T) > 0 **(t) 

( y(T) = 0 



Optimal Control 

! 1 for 0 £ t £ t^ 

0 for t^ < t £ T 



Domain of Controllability 
a l b l y 0 < s2 + (R"l)(b 2 x°) 2 

a l b l y 0 ^ s2 + B ( b 2 x 2^ 2 



( X l^ b l^ = ° 

C 2 X 2 (T) = ° 

V y (T) > 0 

( X l ( ' t 2^ > ° 

C A x 2 (T) = 0 4>*(t) 

( y(T) = o 



! 1 for 0 s t s tj 
0 for t 1 < t ^ T 
1 for 0 ^ t ^ t 2 
- 0 for t 2 < t £ T 



a l b l y 0 > s2 + ( R -l)( b 2 x 2 ) 2 
a i b i y o ^ R * s2 ” ( b i x i) 2 } 

a l b l y 0 S s2 + A ( b 2 X 2^ 2 



/x 1 (T) > 0 

C |x (T) >0 <j>*(t) = 0 for 0 :£ t £ T 

ty(T) = 0 



a^b^y 2 £ Rs 2 { 1 - 1/z} 

a i b i y o > R ^ s2 ~ ^ b i x i^ 2 ^ 



! x 1 (T) > 0 
x 2 (T) > 0 <f>*(t) 

y(T) = 0 



1 

0 



for 

for 



0 s: t £ T-t^ 
T-t^ < t £ T 



a^b^y 2 > Rs 2 {1 - 1/z 2 } 
a l b l y2 > s +A(b 2 x°) 2 
a l b l y 0 < s2 + B(b 2 X 2 )2 



Definition of Times 



(a) 


C 1 


is 


first t 


such 


that 


(b) 


t 2 


is 


first t 


such 


that 


(c) 


T 1 


is 


determined 


by 


cosh 



x i^ t i^ = °* 

2b 1 x 1 (t 2 )x° + b 2 (x°) 2 = a 2 y 2 (t 2 ). 
r-r— R - <5 

"Y2 t 1 = r^T • 
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upon the current state of the system. The results presented in [7] 
(see also [6]) are in the form of a closed-loop control. For deter- 
ministic systems (one-sided problems) , it is well known that open- 
loop control and closed-loop control yield identical results in 
trajectory and payoff [4]. It is, of course, a simple matter to 
convert the optimal control presented, for example, in Table I into 
a closed-loop control. However, for optimal control of stochastic 
systems it is well known that open-loop and closed-loop controls do 
not yield the same return. In Appendix I we consider a stochastic 
version of (1) and consider there a closed-loop control. 

It also seems appropriate to note the following for the 
quantities A and B which appear in inequalities defining various 
domains of controllability: 

(a) for 0 <: 6 < R - >/R(R-l) , 

we have A > B > 0, (2) 

(b) for 6 = R - vk(R-l) , 

we have A = B = 0, (3) 

(c) for R - /R(R-l) < 6 < 1, 

we have A < B < 0. (4) 



Furthermore, let us recall that 
Section 3) . 

A = A(R, z) 



A and B are defined by (see also 



R(z 2 -1) - z 2 

U^lT 2 ’ 



(5) 



B = B (R, z) 



R(z 2 -1) - z 2 
72 



( 6 ) 
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R ~ 6 

The quantity z, defined by z = relates A and B to 6. 

K — 

For Case (3) : 6 = R - /R(R-l) , the solution is shown in 

Table II. The reader should note the reduction in dimension (from 

S S 

three to two) for the domain of controllability or C^_, D(C^). 

Case (3) is an important case for understanding the solution to the 

Isbell-Marlow problem. For R - v^(R-l) < <5 < 1, both extremals and 

the optimal control are unique. In other words, the domains of 

controllability do not "overlap." For 0 £ <5 < R - /R(R-I) , 

extremals are no longer unique (i.e. for a point P° in the initial 

state space there may be more than one extremal path leading to the terminal 

surface) . In Case (3) : 6 = R - /R(R-l) , extremals are unique for 

S S 

all terminal states except C^. Moreover, for C^, the optimal control 

is no longer unique: for any P° £ D(C^) any policy as shown in 

Table II with t^ < leads to the same payoff, P^ = - qx 

Furthermore, any initial point P° £ D(C^) or D(C^) or D (C^) with 

a i^l^0 = a l so leads to exactly the same payoff when an optimal 

control is used, i.e. for such P° we have P- = P, = P c = P ? . 

14 3 3 

For Case (4) : 0 £ 6 < R - /R(R-l) , the domains of controlla- 

bility and corresponding extremal controls are shown in Table III. 

S 

The domains of controllability D(C^), D(C^) , D(C^), and D(C<_) 

all "overlap" each other (as careful study of this table will show) . 
(Inequalities such as A(R s z) > B(R,z) for 0 £ 6 < R - /R(R-l) 

(which are essential in such considerations) are given in Appendix A.) 

Hence, step (e) of the general solution procedure presented in 
Appendix A must be applied. The resulting solution after this has 
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Table II. Solution to Isbell-Marlow Problem for 



6 = R - /R(R-l) . 



Nonrestrictive assumption: R > 1, i.e. a^b^ > a ’2° > 2 



Case (3): 6 = R - /R(R-l) where 6 = a^p/Ca^q) 



Terminal State 



(VV"° ( 

C 1 1 x 2 (T) > 0 = 

( y (T) = 0 l 



Optimal Control 

1 for 0 ^ t ^ t., 



Domain of Controllability 
a l b l y 0 < s2 + (R-l)(b 2 x 2 ) 2 



0 for t., < t ^ T a^y 2 ;> s 



XfCti) = 0 



1 for 0 £ t £ t. 



C 2 ) x 2 (T) = ° 4*(t) = 

y(t) > 0 



a l b l y 0 > s2 + (R-l)(b 2 x°) 2 



0 for t < t £ T 



x 1 (t 2 ) > 0 



1 for 0 £ t £ t r 



a l b l y 0 ^ R ^ s2 " (bjX®) 2 } 



C 4 i X 2 (T) = 0 ** (t) = 



y(T) = 0 



0 for t 2 < t £ T a l b i y o ^ s ‘ 



x 1 (T) > 0 



a i b i y o < R ^ s2 “ 



(b lX °) 2 } 



C 5 j x 2 (T) > 0 4>* ( t) = 0 for 0 £ t £ T 

y(T) = o 



a lVo s s2 



x 1 (T) > 0 



1 for 0 ^ t ^ t 3 < t- 



a iVo ■ s2 



C 5 ) X 2 (T) > 0 4*(t) = 

y(T) = o 



0 for t^ < t £ T 



, o , o, /r~ ... 

b 2 x 2 =• b i x i < /&: - b) 



Definition of Times 



(a) t^ is any time such that 0 £ t^ < t^. 



(b) For t^, t^> and t^, see Table I. 
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Table III. Extremals for Isbell-Marlow Problem for 



0 £ 6 < R - /R(R-l) 



Nonrestricitve assumption: R > 1, i.e. a^b^ > a 2^2 

Case (4) : 0 £ 6 < R - /R(R-l) where 6 = a^p/ (a^q) 



Terminal State 


Extremal 


Control 




o 

II 

✓“N 
i — 1 

4-J 

i — 1 

X 


( 1 


for 


0 £ t £ t 


C ! | 


x 2 (T) > 0 


4>*(t) = < 








' y(T) = o 


( 0 


for 


t < t £ T 



Domain of Controllability 
a l b l y 0 < s2 + ( R-1 )( b 2 x 2^ 2 

a l b l y 0 ^ s2 + B ( b 2 x 2^ 2 



( X 1^ C 1^ = 0 

C 2 x 2 (T) = 0 <*>*(»:) 

( y(T) > O' 

{ X l^ t 2^ > 0 

C 4 x 2 (T) = 0 **(T) 

( y (T) = 0 



1 for 0 £ t ^ t^ 

0 for t^ < t £ T 

1 for 0 £ t £ t 2 
0 for t 2 < t £ T 



a l b l y 0 > s2 + ( R- l) ( b 2 x 2^ 2 
a l b i y 0 < s2 + ( R-1 )( b 2 x 2^ 2 

a i b i y o ^ R ^ s2 " ^ b i x °) 2 ^ 

a l b l y 0 * s2 + A(b 2 X 2 )2 



! x (T) > 0 

x 2 (T) >0 <j)*(t) = 0 for 0 t £ T 

y(T) = 0 



a l b l y 0 < s2 + ( R -l)( b 2 X 2^ 2 
a-jb^y 2 <; Rs 2 {1 - 1/z 2 } 

a i b i y o < R ^ s2 _ ( b i x °) 2 } 



^x 1 (T) > 0 

C^ | x 2 (T) > 0 <j>*(t) 

' y(T) = 0 



1 

0 



for 0 £ t £ T-x^ 
for T-t 1 < t £ T 



a l b i y 0 < s2 + ( R-1 )( b 2 x 2^ 2 

a i b i y o < Rs2 ^ “ b / z2 } 

a l b l y 0 > s2 + A ( b 2 x 2 )2 
a l b l y 0 > s2 + B(b 2 X 2 )2 



'1 * 2* 



Definition of Times: for 



and see Table I. 
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been done is presented in Table IV. A major difference between the 
results presented here and those of Appendix A is that (we have 

S 

discovered since June 1971 that) extremals do lead to C<_ . However, 
this gap in our earlier analysis (see Appendix A) has had no effect 
upon the solution which we presented earlier, since it may be shown 
that for an initial point F° (E D(C<_) D D(C^) we have ^ P^_ when 
the corresponding extremal controls are used. Hence, extremals lead- 

S 

ing to may be dropped (as they were previously inadvertently 

omitted) from further consideration. 

For Case (4) : 0 ^ 6 < R - /R (R-l) , the solution is shown in 

Table IV. It is of interest to note that the solutions contain a dis- 
persal surface (see pp. 134-141 in [5]), a solution aspect rarely 
encountered in optimal control problems (see [2] for the only example 
of which we are aware, other than those given in Isaacs 1 book [5]). 

The reader should note that the solution shown in Table IV is identical 
with that given earlier (see Table II of Appendix A) . 

A considerable amount of effort has gone into the study of 
extremals for 0 ^ 6 < R - /r(R- 1) . In Table V we show event (such 
as annihilation of X^, end of battle, etc.) times and payoffs for 
Case (4): 0 ^ 6 < R - /R(R-l) . We omit derivation of these times, 

which is readily done using elementary considerations (such as combi- 
nation of the extremal control with the solution to the force level 
equations) . The solution to the force level equations is consequently 
shown in Table VI. Additionally, the reader should note that if we 
have <j>(t) = constant for all t € [t^jt^], then the following 
"generalized square law" holds 
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Table IV. Solution to Isbell-Marlow Problem for 
osi<R - /rcr-i) . 

Nonrestrictive assumption: R > 1, i.e. a^b^ > a 2 b 2 

Case (4): 0 £ 6 < R - /R(R-l) where 6 = a^p/Ca^q) 



Terminal State 



/ x^tp = 0 

C l] x 2 (T) > 0 

( y(T) = 0 



Region of Initial Force Levels 
a i b i y o < s2 + (R-i) (t> 2 x °) 2 

a l b l y 0 ^ Rs2 ~ R { b i x °[ z2 ( R-1 ) +R 5 /( 2R ) + b 2 x 2^ 2 ^ z2 
for 0 ^ x° < (b 2 x°)/(k b i) 

a^b^y^ £ h(P°,R,z) for 0 :£ x° £ kb 1 x°/b 2 



XiCtf) = 0 

x 2 (T) = 0 
y(T) > o 



a l b l y o > s2 + (R-l )( b 2 x 2 ) 2 



x ] _(t 2 ) > 0 
x 2 (T) = 0 

y ( t) = o 



a l b l y 0 < s2 + ( R -l)( b 2 x 2^ 2 

a l b l y 0 ^ R ^ s2 “ ( b i x i) 2 } 

a l b l y 0 S h(P°,R,z) for 0 :£ x° :£ kb^x°/b 2 



/x x (T) > 0 
C 5 x 2 (T) > 0 
( y(T) = 0 



a l b l y 0 < s2 + (R-l) (b 2 x°) 2 

a i b i y o < R ^ s2 _ ^ b i x i^ 2 ^ 

a i b i y o s Rs2 ” R{ b 1 x ^[ z2 (R -1 ) +R ]/( 2R ) + b 2 x°} 2 /z 2 

for 0 ^ x° < (b 2 x°)/(kb 1 ) 



Explanation of Symbols 



(a) For t^, t^, and see Table I. 

(b) k = [z 2 - R(z-l) 2 ] / (2R) • 
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Table V. Event Times and Payoffs for Isbell-Marlow Problem 
when 0 £ 6 < R - R(R-l) . 

Terminal State C^: x^Ct^) = x 2 (T) > 0, y(T) = 0 



= £n { /a l b l y Q ~ S " + (b 2 X 2 )2 ~ b 2 X 2 } 



^1 



y o ' s 



T = t, + — - — canh- 1 ) ^l b l y ° - * ( V^ , 



- 1 ^ 



b 2 X 2 ^ 



Pa = — 3 ~ /s z + (R-l)(b xp z - a b y 2 
b 2 /R -lxu 



Terminal State C^: x^(t^) = 0, x^CT) = 0, y(T) > 0 



h ■ 



1 tn( SVo - S " + (b 2 1I 2 ) " - b 2*2 J 






y 0 “ s 



= t,, H — tanh 

1 i — : — 1 



b 2 X 2 ^ 









v/a l b l Y 0 " sZ + ( b 2 x 2^ 



Terminal State : x 1 (t 2 ) = x-^T) > 0, x 2 (T) = 0, y(T) = 0 



t 2 : 



for a^y 2 > s 2 



C 2 = 



i y(t 9 ) - /y z (t ) - y z + s z /(a b ) 

1 £n{ ? ? = ^ = } 



•Yi 



y o ' s/v ^X 



for a l b l y 0 < s2 



C 2 = 



l/ y 2 (t 9 ) - y 2 + s2/(a b ) - y(t ) 

1 £n{ ^ 0 — } 



"Yi 



for a 1 b 1 y 2 = s, 



s/^?T" y 0 
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Table V. (Concluded) 



where 



fco 

y (t 2 ) = J {(2R-l)b 2 x° + 2/s z + R(R-l) (b 2 x°) z - a^yj^ } , 



also 



T = t 2 + 



1 _i ( Rb 2 x 2 + ^ + R(R-l) (b 2 x°) z - a^y 



v ^?2 



cosh 



(R-l)b 2 x° + /s z + R(R-l) (b 2 x°) z - a^yj^ 



P 4 = b^R ^ b 2 X 2^ R_1 ^ + ^ S ' 1 + R ( R_1 ) ( b 2 x 2^ Z - a l b l y 0^ 



Terminal State C^: x^(T) > 0, x^CT) > 0, y(T) = 0 
(no switch: <j>*(t) = 0 for 0 ^ t ^ T) 



I = _ i _ tanh - 1 { ^ y o 



) — 7 — ' b x° + b x 

/ a 2 b 2 11 22 



o 1 



Terminal State C^: x^(T) > 0, > y(T) = 0 



switch in tactics: <f>*(t) = 



1 for 0 ^ t ^ T - T- 



0 for T - x 1 < t £ T 



1 , -1 

T i ‘ 7 = cosh fcp 

' V 2 



T = T, + 



1 ( (/R(z z -1) -z) ( /a. 1 b 1 y Q +s) 

-i &n < — — ' 

2/a^b^ ( (/a^b^ y^-s) (^(z 2 -!) + z) 



R 5 = {-z( b 2 x°) | /(z z (R-l)-R)(a 1 b 1 y z -s^) } 



hJ O 
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Table VI. Solution to Force Level Equations for 
Two-on-One Combat. 

dx„ 



Combat Equations: 



di *v 

dx 

— = -(l-*)a 2 y 



with x^(t= 0 ) = x^, 



with x 2 (t= 0 ) = x 2 . 





= — h x. 


— b„x„ 


dt 


1 1 


2 2 


<j>(t) = 


constant 


for t 


* a i( 


( ' b l X l +b 2 X 2^ 


[ 4> a l b 1 + ( 1— ) a 2 b, 



cosh/fia^b^ + (l-<fi)a 9 b 2 t 



^ b l X l +b 2 X 2^ 



/ 4 > a 1 b 1 + <1— 4») a 2 b 2 



sinh/cjia., b_ + (l-<{>)a„b„ t — - T r : — 71 r — r — r 

11 2 2 [<j>a 1 b 1 + (l-,j,)a 2 b 2 ] 



, s o [ < b l X l +b 2 X 2> 

x 2 (t) x 2 + (1 ^)a 2 j + ( 1 — 4 > ) a 0 b 0 ] 



2 2 



cosh/ ^b ^ + (1 -V) a ^b 2 " t 



(b 1 x°+b 2 x°) 



/(Jia^b^ + (l-(f))a 2 b. 



si„h/*a 1 b 1 + (l-«a 2 b 2 t - [ 4 ,^^ + (i : Wa , b j 



2 2 



y(t) = y Q cosh/(J)a 1 b 1 + (l-<j>)a 2 b 2 t 



(b 1 x °+ b 2 x 2 ) 
v/< ^ >a i b l + (l -< l > ) a 2 b 



sinh/(j>a^b^ + ( 1 — c}> ) a 2 b 2 t 
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C 2 (t=t x ) - 5 2 (t=t 2 ) = (<|)a 1 b 1 + (l- ( j.)a 2 b 2 Hy 2 (t=t 1 ) - y 2 (t=t 2 >} (7) 

where 

£(t) = b 1 x 1 (t) + b 2 x 2 (t). 

This generalized square law has been used (along with the extremal 
control to C /) to express = ry(T) - px^T) - qx 2 (T) in the form 
shown in Table V. 

As a final check on our theoretical developments, we had some 
numerical computations done for Case (4) . The results shown in Table 
V have been used in this work. We would like to thank a M.S. thesis 
student, Robert L. Powers, LT , USN, for his efforts in this area. 

All Powers* computations have supported the solution shown in Table 
IV. For example, let us examine the results of computations for typical 
data such as that shown in Table VII. For this data, we have 6=0 
and R = 2. Extremals lead to C^, C^, and from P°. For the 

extremal to C^, we have t^ = 4.26 minutes while T = 18.99 minutes 
and P^ = -217.72. For the extremal to , we have T = 9.96 

S 

minutes and P<. = -149.92. For the extremal to C^, we have T - = 

S 

3.64 minutes while T = 16.81 minutes and P<- = -221.43. Considering 
Table IV, it is readily shown that P° = (100,100,152) is such that 
the optimal path leads to C^, as these numerical computations bear 
out. Notice that for the optimization problem (1) which we are con- 
sidering and the input data shown in Table VII, making the battle last 
longer (by firing more at X^) does not result in more value for 
survivors (from Y*s standpoint). Finally, because in our earlier 
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Table VII. Time History of Force Levels 

S 

for Extremal Leading to 

x° = 100, x° = 100, y Q = 152 

a^ = 0.2 - cas./(min. x Y unit) 

a^ = 0.1 X - cas./(min. x Y unit) 

b^ = 0.1 Y - cas./(min. x unit) 

= 0.2 Y - cas./(min. x unit) 

p = 0.0, q = 5.0, r = 10.0 



time, t 


x 1 (t) 


x 2 (t) 


y(t) 


0.00 


100.00 


100.00 


152.00 


0.36 


89.19 


100.00 


144.91 


0.73 


78.88 


100.00 


138.21 


1.09 


69.04 


100.00 


131.88 


1.46 


59.65 


100.00 


125.89 


1.82 


50.69 


100.00 


120.24 


2.19 


42.12 


100.00 


114.91 


2.55 


33.94 


100.00 


109.88 


2.91 


26.11 


100.00 


105.14 


3.28 


18.61 


100.00 


100.69 


3.64 


11.43 


100.00 


96.50 


4.96 


11.43 


88.22 


82.62 


6.28 


11.43 


78.18 


70.18 


7.59 


11.43 


69.69 


58.95 


8.91 


11.43 


62.60 


48.75 


10.23 


11.43 


56.81 


39.40 


11.54 


11.43 


52.20 


30.73 


12.86 


11.43 


48.69 


22.59 


14.18 


11.43 


46.23 


14.85 


15.50 


11.43 


44.77 


7.36 


16.81 


11.43 


44.29 


0.00 
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work (which is reported in Appendix A) we had erroneously concluded 

g 

that no extremals led to C 5 for 0 s; 6 < R - /r(R- 1) , we also have 

g 

verified that extremals do lead to C,_ by examining the time history 
of the force levels computed according to the extremal control. An 
example of the results of such computations is shown in Table VII. 

5 . Necessary Conditions of Optimality . 

The solution shown in Tables I, II, and IV has been developed 
according to the solution procedure which we outlined in Appendix A. 

This procedure is based upon synthesizing extremal trajectories from 
the basic necessary conditions of optimality. Thus, it seems appro- 
priate to say a few words about these. 

Since we have developed in Section 4.b.(l) of Appendix E 
the basic necessary conditions of optimality for (1) , we will not 
repeat details again but refer the reader there for all the details. 

As we saw in Appendix E, the Isbell-Marlow problem requires use of 
results (boundary conditions for the dual variables, necessary con- 
ditions for it to be optimal to have x^(t) = 0 for a finite interval 
of time, corner conditions) from the theory of SVIC. In the original 
pioneering work of Isbell and Marlow [6] these results were not used, 
since they had not yet been discovered and published in the mathe- 
matics literature. In our earlier work (see Appendix A in this report 
and also Appendix A of [8]) we had followed some heuristic reasoning 
of Isbell and Marlow [6] on such points. 

Let us now summarize the main results of Section 4.b.(l) of 
Appendix E. We saw that in order for it to be optimal to have x^(t) = 0 
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for a finite interval of time we must have 



a l b l * a 2 b 2 . 



( 8 ) 



At entry at t^ to a constrained subarc on which a. = 0, we have 
(from the corner conditions) 



P l (t l ) a ± P 2 (t l ) ’ 



P i (t 1 ) = ) for i = 2,3, 



(9) 

( 10 ) 



where t, denotes a left-hand limit. Furthermore, for C. we have 
1 4 

that 

a l P 

< (-p (t=T)) ^ q. (11) 

a 2 Z 

The latter of these two inequalities was conjectured in Appendix A 
(in June 1971) to be a property of an extremal. Our recent work using 
the theory of SVIC has confirmed this conjecture. 



S 

6 . Existence of Extremals Leading to . 

In our earlier work [9] we erroneously concluded that 

g 

0 <; 6 < R - /r(R- 1) there were no extremals leading to . 
now retrace our earlier development in a correct manner. 

There are two subcases for entry to C,- . We consider 
extremals for which 



for 

Let us 
the 



/ 1 for 0 ^ t £ T - 

<j,*(t) = I 

V 0 for T - < t ^ T . 



( 12 ) 
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c 

For convenience, let us say that such an extremal leads to C^. 
Combining the above extremal control with the generalized square law 
(7) , we obtain 

R(b 2 x 2 (T)) 2 + 2Rb ] b 2 x 1 (T-T 1 )x 2 (T) + (b^ (T-^) ) 2 

+ 2(l-R)b ] b 2 x 1 (T-T 1 )x 2 + a^y 2 - s 2 + (1-R)(b 2 x°) 2 = 0. (13) 

A backwards integration of the state equations of 0 & x £ with 
x 2 (t=0) = x 2 (T) and y(x=0) = 0 yields that 

b 2 X 2 = b 2 x 2^ z + b 1 x 1 ( T ~T 1 ) (z-1) , (14) 

since x 2 (t=T-x^) = x°. It should be noted that in deriving (14) we 
have assumed that x^(T-x^) > 0. (13) and (14) may be combined to 

yield 

(b 2 x 2 (T)-b 2 x°) 2 = (a 1 b 1 y 2 -s 2 )/A, (15) 



and hence we require that the right-hand side of (15) be positive in 

S o S 

order that be reached. Thus, for P 6 D(C,_) we' must have 



a l b l y 0 •= 3 2 


for 


A < 


0, 


(16) 


a lVo = 32 


for 


A = 


0, 


(17) 


a l b l y 0 > 32 


for 


A > 


o, 


(18) 


where the reader should keep (2) 


through 


(4) 


in mind. 





Now when A < 0 (corresponding to R - /R(R-l) < 6 < 1) , (15) 



may be solved to yield 



The requirement that x 2 ^) > ® then leads to 



a l b l y 0 > s2 + A (b 0 x°) 2 , 



2 2 



( 20 ) 



where we have made use of (5) . Combination of (14) and (19) yields 
that 



, T _ , = o /s Z a l b i y p 

> l X l( T V b 2 X 2 + Z / R - z z (R-l) ' 



( 21 ) 



The 



requirement that x^(T-x^) > 0 then leads to 



a l b l y 0 < s2 + B(b 0 x°) 2 , 



2 2 



( 22 ) 



where we have made use of (6) . Furthermore, the time T - x^ may be 



determined by solving the following equation (see Table VI) 



y(T-x^) = y^ cosh/a^b^ (T-x^) — sinh/a^b^ (T-x^) to yield 






A 



T - Xi = 



In 



a l b l y 2 ( T_I i) - a 1 b 1 Yo + - v/a 1 b 1 y( T -' r 1 ) 






s - 'Vi y o 



(23) 



The requirement that the quantity in brackets is greater than one leads 
to 

y Q > y(T-x x ). (24) 

Now, the generalized square law (7) may be combined with the extremal 
control (12) and (14) to yield 
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>/ a^b~ y (T— t^) = /R(z z -1) 



F - a iVo 

/ R - z z (R-l) 



(25) 



(24) and (25) may be combined to yield 



a l b l y 0 > Rs2 ^ “ 1 / z2 ^- 



(26) 



Furthermore, (23) and (25) may be combined to yield 



, ( (z-/R(z z -l) ) (s+i/a b y n ) i 

T-t - — ln{ — >. (27) 

2 v/a i b l ' (s-/a^b^ y Q ) (z+/R(z 2 -1) ) 

Finally, (19) may be used to show that the payoff corresponding 

to use of the extremal control ( 12 ) is given by 



P 5 * { _z (\^) b 2 x 2 + R / ( R ~ zZ ( R ~D ) (s^-a^y*) | • (28) 

When A > 0 (corresponding to 0 £ 6 < R - /r(R- 1) ) , we must 
have a^b^y^ > by (18) so that analysis similar to the above 

readily yields results similar to ( 20 ) , ( 22 ) , and (26) but with the 
inequalities reversed (see Table III) . In our previous work, we had 
erroneously concluded that this case was impossible. 

Next, we consider the extremals for which 

<J>*(t) = 0 for 0 £ t £ T. (29) 

Let us adopt the terminology that such an extremal leads to with 

payoff P^. We omit discussion of results previously derived in 
Appendix A. In that appendix we showed that 
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b 2 X 2 (T) = " b l X l + /s " “ a l b i y O /R> 



(30) 



Using (30), we may express P^ in the following form 



P = -3- 
5 b 2 



{z(^) b l x ° - - a 1 b 1 yg/R } . (31) 



7 . Proof of a Dominance Theorem . 

In this section we show that for 0 £ 6 < R - /R(R-l) we need 

g 

not consider extremals that lead to since the payoff associated 

with such an extremal path is exceeded (or dominated) by the payoff 
associated with an extremal leading to C,_ . Hence, our previous over- 

g 

sight on extremals leading to is of no consequence. 

Our main result is stated below as Theorem 1. It is convenient, 
however, to first explain our terminology in a precise fashion. To 
be precise, then, we have 



g 

Definition 1: We say that an extremal leads to 

when we use the extremal control 

( 1 for 0 <; t £ T - t^, 

0 for T - T 1 < t £ T, 
and reach the terminal state x^(T) > 0, 

x 2 (T) > 0, y(T) = 0. (t is given in 

Table V.) 



We denote the payoff associated with an extremal leading to C,. by 

g 

P,-. We have that for 0 ^ 6 < R - /R(R-l) 



P 5 = hi {' Z ^ b 2 X 2 + I ^(z z (R-l)-R)(a 1 b 1 y 0 -s^)}. 



(32) 
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c 

Furthermore, the domain of controllability for C,., denoted by 

S 

D(Cj.) , is given by 
for 0s{ <R- /R(R-l) 

D(C^) = {P° = (x°,x°,y Q ) | a^b^y^ s: Rs 2 (l-l/z 2 ) and 

s 2 < s 2 + B(b 2 x°) 2 < a^^y 2 < s 2 + A(b 2 x°) 2 }. (33) 

We also have 

Definition 2: We say that an extremal leads to when 

we use the extremal control 



4>*(t) = 0 for 0 £ t <; T, 

and reach the terminal state x^(T) > 0, 
x 2 (T) > 0, y (T) = 0. 

We denote the payoff associated with an extremal leading to by 

Pj>. We have that for 0 ^ 6 < R - /R(R-l) 



P 5 = HtM - ^ - a lVo /R } 



(34) 



Furthermore, the domain of controllability for denoted by D(C<-), 

is given by 

for 0 it 6 < R - /R(R-l) 

D(C 5 ) = (P° = (x°,x°,y 0 ) | a^y 2 Rs 2 (l-l/z 2 ) and 

a i b i y o < R ^ s2 - ^ b i x i^ 2 * ^ • O 5 ) 
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To be precise, we must also add the following condition to (33) and 
(35) 

a l b l y 0 < s2 + (R-D (b 2 x°) 2 . (36) 

Since (36) does not enter into the proof of Theorem 1, we have (for 
convenience) omitted it. 

We now state the main result of this section as Theorem 1. 

THEOREM 1: Assume that 0 <; 6 < R - /R(R-l) . (This is 

equivalent to z > /R/ (R-l) . ) Then P° = 

(x°,x°,y Q ) 6 D(C,-) f| D(C^) implies that 

S o 

P,. k P,. with equality holding only when a ^b;L y o 

Rs 2 (1-1/z 2 ) . 



PROOF : 

A. First we will show that 

s2 < a l b l y 0 < Rs2 ( 1 ~1/z 2 ), (37) 

and 

< 38 > 

S o S 

imply that P^ < P^. For P 6 D(C^) f) D(C^) we have (postponing 
for now the case when equality holds) 

a l b l y 0 < Rs2 U-1/z 2 ), (39) 

which implies that 

R(z 2 -1)s 2 - z 2 a^b^y 2 



> 0 . 



(40) 
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Squaring (38), we may rearrange the result to obtain 

(2Rszi/s z - a^b^g/R) 2 < {R(z 2 +1)s 2 - z 2 a^b^y^} 2 . (41) 

By observing that 1 - 1/z 2 < 1 + 1/z 2 and considering (39), it is 
readily seen that 

R(z 2 +1)s 2 - z 2 a^b^y 2 > 0, (42) 

so that we may take square roots in (41) to obtain 

2Rsz/s 2 - a^b^y 2 /R < R(z 2 +l)s 2 - z 2 a^b^y 2 . (43) 

Now it is easily seen that (41) may be rearranged to yield 

(z 2 (R-1)-R) (a^b^y 2 -s 2 ) < {z(R-l)s - R/s z - a^^/R} 2 . (44) 

The conditions s 2 < a^b^y 2 and z 2 > R/ (R-l) are readily combined 
to yield 

R 2 (s 2 -a^b^y 2 /R) < z 2 (R-l) 2 s 2 . (45) 

Recalling that for P° € D(C,.) D D(C^) we have a ]_b]_yg < Rs 2 (l-l/z 2 ) 
< Rs , we may extract square roots in (45) to obtain 

z(R-l) s - r/s 2 - a^b^y 2 /R > 0. (46) 

Considering (37) , (38) , and (46) , we may extract square roots in 
(44) to obtain 

£ Az 2 (R-1)-R) (a-jbjyQ-s 2 ) < z(^p) (b 1 x°+b 2 x°) - /s 2 - a^^/R. 



(47) 
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Some additional manipulations on (47) now readily yield that (37) and 

S 

(38) imply that P^ > P,.. 



B. Next we show that 



a.^.^ = Rs 2 (1-1/ z 2 ) , 



(48) 



implies that P<_ = P<_. It is readily shown that (48) leads to 



/(z z (R-l)-R)(a b y z -s ) = f (z 2 (R-l)-R) 
1 I U z 



(49) 



Now (32) and (49) may be combined to yield 



> s = -SI- 
'S b„ 



HtM-I} 



(50) 



Furthermore, (48) also leads to 



s 2 - a 1 b 1 y 2 /R = s 2 /z 2 . 



(51) 



which may be combined with (34) to yield 



P = 

5 b 2 






(52) 



Consideration of (50) and (52) shows that (48) implies that P^ = P^, 



By A. and B. above, Theorem 1 is proved. 



Q.E.D. 



8 . Development of Conditions for Extremals to Reach . 

Another topic for which our previous discussion [9] was 
inadequate was the development of conditions for extremals to reach 
C^ . (Previously, we heuristically developed (66) and had not shown 
that (70) was a necessary condition.) 
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Let us define 



w = w(p 2 (t=T)) 



a l ( b i p 2 Ct =T )+ b 2 p ) 

P 2 (t-T)(a lV a 2 b 2 ) • 



which we may write as 






where 



(53) 



Q = (-p 2 (t=T))/q. 



(54) 



Then recalling (11) (which was a necessary condition of optimality) , 
we have that 
for 6 < Q £ 1, 

we have 1 < w(Q) £ f = z, (55) 

K— 1 

where w(Q) is a strictly increasing function of Q arid w(Q=6) = 1. 
Now 

w = cosh/a 2 b 2 (T-t 2 ) , (56) 

and 

z = cosh/a^b^" t = cosh/a^b^ t^(C^). 

Since cosh x is a strictly increasing function, (55) and (56) readily 
yield 

T - t 2 £ « T^C®). (57) 

Now, the extremal control is 

, 1 for 0 £ t £ t 2 , 

4>*(t) = < 



0 



for 



t 2 £ t £ T, 



(58) 



F-30 



where is the smallest t such that 

a l b l y2 (t2 ) = R ^ b 1 x 1 ( t 2 ^ +b 2 X 2^ 2 ” (b 1 x 1 (t 2 )) 2 }. (59) 

In [9] we showed that the extremal control (58) and the generalized 
square law (7) yield 

b lXi(t 2 ) = b 2 x 2 (R - l) + ^ + R(R-l) ( b 2 X 2^ Z " a l b i y 0 ' 

Clearly we must have b^x^(t 2 ) ^ b l x l’ anc * tb ^- s ma y be combined with 
(60) to yield 

b 2 x°(R-l) <; b xj. (61) 

A backwards integration of the state equations using (58) yields 
for t 2 £ t £. T 

b 2 x 2 ( t ) = b i x i^ t 2^ cosh ' /a 2 b 2 ^ T-t ^ " 1 ^» (62) 

so that 



b 2 X 2^2^ = b 2 X 2 = b l X l^ t 2^ COShv/a 2 b 2 ^ T-t 2^ ” ' ( 63 ) 



Then (60) and (63) may be combined to yield 



T - t 2 - 



cosh 



_ ± ( Rb 2 X 2 + + R(R-l) (B 2 x°) z - a^y 2 






(R-l)b 2 x° + /s 2 + R(R-l) (b 2 x°) 2 - a^^ 2 ' 



Recalling that 



( 6 ‘ 



h - T l(C p 




5 



(65) 
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we may combine (57) and (64) to obtain 

a l b l y 0 ^ s + A(b 2 x°) 2 . (66) 

Now, since <J>*(t) - 1 for 0 ^ t ^ we have by the generalized 

square law (7) 

a l b l y 0 “ a l b l y2(t 2 ) = s2 ~ (b 1 x 1 (t 2 )+b 2 x 2 ) 2 . (67) 

We may combine (59) and (67) to obtain 

a l b l y 0 = (R-l) { (b 1 x 1 (t 2 )+b 2 x°) 2 -s 2 } + R{ (b^x°) 2 - (b^^)) 2 } 

+ R{s 2 - (b^ x °) 2 } . (68) 

We will prove as Proposition 1 that 

R{ (b x x°)2 - (b 1 x 1 (t 2 )) 2 } - (R-l){s 2 - (b 1 x 1 (t 2 )+b 2 x°) 2 } k 0, (69) 

so that (68) and (69) yield 

a l b i y 0 k R ^ s2 ~ (b^®) 2 }. (70) 

It remains to prove the following proposition 

PROPOSITION 1: A necessary and (with appropriate additional 

assumptions) sufficient condition for (69) 
to be true is that b^x^R-l) £ b^x°. 

PROOF: We prove necessity only (sufficiency follows by reversing the 

chain of arguments) . 
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It is readily shown that (69) may be rearranged to yield 

(bjX®) 2 - (b 1 x 1 (t 2 )) 2 :> 2(R-l)b 2 x°{b 1 x° - b 1 x ;L (t 2 )}. (71) 

Now (71) may be factored and (assuming that b^x° > a common 

term cancelled on both sides to yield 

b x x° + b lXl (t 2 ) :> 2(R-l)b 2 x° (72) 

Combining (60) with (72), we obtain 

/s* + R(R-l)(b 2 x°)^ - a^yg ;> b 2 x°(R-l) - b^®, (73) 

and the necessity of the proposition is proved. Q.E.D. 



The time t^ in (58) may be explicitly determined by solving 
the equation (see Table VI) 



y(t=t 2 ) = y Q cosh/ a^b^ t 2 sinh/a^b^ t 2> (74) 



and this yields 



y(t 2 ) - /y z (t 2 ) - Yq + s z /(a 1 b 1 ) 

y Q - s//I^ 

Since the quantity in brackets must be ^1 (for all cases, i.e. 
a 1^1^0 > y ~ 9 < s ^) > we must have 

y Q :> y(t 2 ) > 0. (76) 





Now (59) and (60) may be combined to yield 
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y 2 (t 2 ) = {(2R-l)b 2 x° + 2/s 2 + R(R-l)(b 2 x°) z - a^y 2 } , 



(77) 



so that when this is combined with (76) , we obtain 



a 2 b 2 y 0 



^o- - (2R-l)b 2 x 2 j :> 2/s 2 + R(R-l)(b 2 x°)* - a^yg . (78) 



Hence, we must have 



a l b l y 0 ^ R(2R-1) (b 2 x°) 2 . 



(79) 



If (79) holds, then squaring both sides of (78) again leads to (70). 
Moreover, (61) and (70) yield (79) as Proposition 2 shows. 



PROPOSITION 2: Assume that 



v , o , o 

(R-l)b 2 x 2 «: b^. 



Then 



ap^y 2 ^ R{s 2 - (b^x°) 2 } implies that 
a l b l y 0 ^ R ( 2R_1 ) ( b 2 X 2^ 2 * 



PROOF: We may manipulate (R-l)b 2 X 2 ^ b^x° to °^ ta ^ n 



(2R-1) (b 2 x°) 2 <: 2b 1 x°b 2 x° + ( b 2 x°) 2 . 



Hence 



R(2R-1) ( b 2 x°) 2 ^ R(s 2 - (b-j^xp 2 } , 



°'>2i 



and whence the proposition. 



Q.E.D. 
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9 . Analysis of Dispersal Surface . 

For 0 £ 6 < R - /R(R-l) , the domains of controllability 
for all the terminal states are shown in Table III. It is easily 
seen that many of these domains overlap each other (the inequalities 
(47) through (53) of Appendix A are useful in such considerations) . 
Hence, we must use "considerations in the large" to select the optimal 
policy from among the candidate extremal policies. This is essentially 
step (e) of our general solution procedure outlined in Section 4. of 
Appendix A. 

Our "considerations in the large" have led us to discover a 
new aspect of the Isbell-Marlow problem [6] : the solution contains 

dispersal surfaces for 0 £ 6 < R - /R(R-l) . A dispersal surface (see 
pp. 134-141 in [5]) is a surface only away from which optimal paths 
lead. It is the locus of points such that the payoffs from two families 
of extremals are equal. 

Let us now show how the dispersal surface which separates 
optimal paths leading to and also is explicitly determined. 

We have previously summarized results in [9] (see Appendix A in this 
report), but we present the details here. 

It seems appropriate to firmly establish our notation first, 
however. To be precise, then, we have 

Definition 3: We say that an extremal leads to when we 

use the extremal control 

! 1 for 0 £ t £ t^, 

0 for < t £ T, 
and reach the terminal state x^(T) = 

x 1 (t 1 ) = 0, x 2 (T) > 0, y (T) = 0. 
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We denote the payoff associated with an extremal leading to by 

P^. We have that 

p q = ^s 2 + (R-l) (b xp z - a b y 2 . (80) 

b 2 & z z 1 1 u 

Furthermore, the domain of controllability for C^, denoted by D(C^), 
is given by 



Dtt^) = {P° = (x°,x°,y 0 ) | a^yg < s 2 + (R-l)(b 2 x°) : 



and 



a l b l y 0 ^ s2 + B(b 2 x°) 2 }. 



(81) 



We refer the reader to Section 7 for the corresponding notation 
5 

and results for C<_ and C^. However, let us be more precise in 
(35) ; we now write this as 
for 0 £ <S < R - /R(R-l) 



a l b l y 0 < s2 + (R-l) (t> 2 x°) 2 , 



D(C 5 ) = (P° = (x°,x 2 ,y Q ) 



a^y 2 £ Rs 2 (l-l/z 2 ), and 
a 1 b 1 y 2 < R{s 2 - (bjX®) 2 }.} . (82) 



Considering (33) , (35) (or equivalently (82) , and (81) , we see 

Q 

that D(C^) f| D(C<_) n D(C,_) is nonempty (see inequalities (47) 
through (53) of Appendix A), However, Theorem 1 tells us that we need 

S 

not consider further extremals leading to Thus, we must determine 

optimal paths from extremals for P° G D(C^) 0 D(C^). 

Now by (81) and (82) P° such that = s 2 + 8 ^ 2 * 2) 2 

clearly belongs to D(C^) flD(C^). (The reader should recall (2): 

B > 0.) We begin by stating and proving Theorem 2. 
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THEOREM 2; Assume that 0 £ <5 < R - /R(R-I) . Then for 
a l b l y 0 = s2 + B ( b 2 x 2^ 2> we have P 5 > P l # 



PROOF : 

A. Let us first observe that 6 < R - /R(R-l) implies that 



z 



^ > 




(83) 



B. Now it is readily shown that (83) leads to 

z2 (¥hV?) 2 + 2b l x l b 2 X 2 > s2 " ( b 2 x 2^ 2, (84) 

We may add the term ( b 2 x 2^ 2 ^ z2 t0 k° tb sic ^ es °f (84). This makes 
the left-hand-side of (84) a perfect square so that extracting square 
roots, we obtain 

b x o 

Z (\^ b l X l + > ^ 2 ' r + B( ‘ b 2 X 2^ 2 ^ ’ ( 85 ) 

where we have made use of the definition (6) of B. Recalling that 
we are considering the case when a^b^y 2 = s 2 + BCb^x^) 2 ) we see 
that (85) leads to 

b x° 

2 ) b i x i “ v/s2 " a i b i y o /R > " • (86) 



Hence, considering (34) for P<_, we have 

«, (b 2 x 2> 



P 5 * 



z 



(87) 
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C. For a^b^y 2 = s 2 + B (b2 X 2^ Z ’ it is readily shown by (80) that 



reduces to 



P ..iiV? 

1 b 2 z 



whence the theorem follows. 



( 88 ) 

Q.E.D. 



Now for 0 :£ 6 < R - /R(R-l) , we showed in Appendix A that 

s 2 + B(b 2 x°) 2 < Rs 2 (l-l/z 2 ) . (89) 

Hence, P° such that = Rs 2 (l-l/z 2 ) clearly belongs to 

D(Ci) 0 D(C,_) . We now state and prove Theorem 3. 

THEOREM 3: Assume that 0 £ <$ <R - /R(R-l). Then for 

a i b i y o = Rs2 ( 1_1 / z2 ) and p ° € d^) n D(C 5 ), 

we have . 



PROOF : 

A. Again we observe that 6 < R -/R(R-l) implies that 



z 



2 > 




(90) 



B. We may readily obtain from (90) that 

^^(b^) 2 > (s-b 2 x°) 2 , (91) 

which is easily further manipulated upon to yield 

^ fe 2 + ^ R “ 1 ^ b 2 X 2^ 2 " Rs 2 (l-l/z 2 ) } . (92) 

Now for 6^0, we readily have 
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, o cR— l'i, o 

Vi * z hn b i x i 



(93) 



By (82) we have that for P £ D(C<_) 



a l b i y 0 <R k 2 ■ (b-j^Xj^) 2 }, 



(94) 



Since we are assuming that 



a l b l y 0 = Rs2 U-1/z 2 ) , 



(95) 



we may obtain from (94) that 



,os 

Vi 5 7 • 



(96) 



so that (93) and (96) yield that 



s (R-l'w o 

z z hP b l x l 



(97) 



Considering (97) , we may extract square roots in (92) to obtain 

^ = z (^R^) b l x l > ~^ sZ + (R-l)(b 2 x°) z - Rs (1-1/z 2 ) . (98) 

/ R 



Recalling the expression (80) for P^, we readily see from (98) 



that 



p i =• t; { z ^Vi -f}. 



(99) 



C. Since we are assuming that (95) holds, it is readily shown 



using (34) that P<_ reduces to 



p = -a- 

5 b 2 



{ z ( s irN x i • !}• 



( 100 ) 



whence the theorem follows. 



Q.E.D. 
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Thus, by Theorems 2 and 3 we have shown that for 0^6 <R - / R(R-l) 
(or, equivalently, for /] R./(R-1) < z ■£. R/(R-1)) 

k. P 5 > P x for a^y 2 = s 2 + B(b 2 x°) 2 , 
or 

(P 1 -P 5 ) <0 

and 

B * P 1 51 P 5 f ° r a l b l y 0 = Rs2 ( 1-1 / z2 )> 
or 

(P 1 -P 5 ) > 0 

where we also assume that P° € D(C^) f) D(C^) . We also note that 
s 2 + B(b 2 x°) 2 <Rs 2 (l-l/z 2 ) for 0 £ 6 <R - /R(R-l) . The locus of 
points for which P^ = P,. (or P^ - P^ = 0) is the dispersal surface . 

It is uniquely determined, since we always have 

a “V ( P i -P c) > 0- (101) 

9 y 0 15 

We now prove (101) as Theorem 4. 



THEOREM 4 : Assume that x^ > 0 and R > 1 . Then 



air < p r p 5> > °- 



PROOF : 

A. Using (34) and (80) , it is easily computed that 



VlV 


(. F 1 + 


- b 2 /R , 


( / Rs * " a lVo 



3 y 0 15 /s 2 + (R-l)(b 2 xj) 2 - a^y 2 



( 102 ) 
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B. 



Now for x^ > 0, we have 



(R-lHb^x^) 2 < (R-l)s 2 for R > 1, 



(103) 



We may rearrange (103) and perform further manipulations to obtain 



r- 



+ (R-l)(b 2 x°) z - a 1 b 1 y^ 



< 1 , 



(104) 



whence the theorem follows. Q.E.D. 

Our results (Theorems 2, 3, and 4) are summarized in Table VIII. 
The dispersal surface is the locus of points such that P^ = P^. 
Equating (34) and (80) , we find the equation of the dispersal surface 




a l b l y 0 = Rs2 ” R { b i x °[ z2 ( R_1 ) + R]/(2R) + b 2 x°} 2 /z 2 , (105) 

for 

0 £ x° c 2Rb 2 x°/ (b^tz 2 - R(z-l) 2 ]}. (106) 

Expression (106) must also be considered, since (94) must also hold 

for extremals leading to C^. Considering Theorem 4 and (105) (with 

(106)), we readily obtain the corresponding results shown in Table IV. 

For 0 £ 6 < R - / R(R-l) , there may also be a dispersal surface 

separating optimal paths which lead to and . As noted in 

Appendix A, the details are much messier than those for S , and 

1-5 

an explicit expression has not been obtained for S 

U l-4 
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Table VIII. Summary of Results on Dispersal Surface 
which Separates Optimal Paths Leading to and C,,. 

For 0 S { < R - / R(R-l) (i.e. for / -~r- < z £ ~~ ) 

v K— 1 R — JL 



1. (P,-?,;) <0 for a^y 2 = s2 + B ( b o x °) 2 



1 5 



2 2 



2. (P 1 ~P 5 ) > 0 for a i b ]7o = Rs2 U-l/z 2 ) 



3. ^ — (P,-?,.) > 0 for x° > 0 and R > 1 

9 y 0 1 5 1 



Let us note that 



s 2 + B(b 2 x°) 2 <Rs 2 (1-1/z 2 ) 



Then 1, 2, and 3 imply that there exists a uniquely determined surface 

such that P = P . Denote this surface as S 

1 5 D l-5 



s c {d(c ) n d(c )}. 
1-5 



Clearly 
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10 . Proof of a Useful Inequality , 

The following lemma has been useful in developing the domains 
of controllability for and . 

LEMMA 1: Assume that 0 ^ y ^ 1 and x ^ 1. Then a necessary 

and sufficient condition for 

tanh ^ y ^ cosh ^ x to hold is that 



PROOF: We recall that 



tanh 1 y = y > (107) 

and 

cosh ^ x = &n (x+/ x 2 -l) . (108) 



A. Proof of necessity: 

Assuming that tanh ^ y ^ cosh ^ x, we may use (107) and 
(108) to obtain 



^ n (l-y) ^ £n(x+/x z -l) 2 . (109) 

Since the logarighm is a strictly increasing function, (109) yields 
that 

1 + 7 -^- ^ 2 x 2 + 2 x/x z -l . ( 110 ) 

1-y 

Further manipulation on (110) lead to 

7 ^— - x 2 ^ x/x z -l . (Ill) 

1 -y 
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Since the right-hand-side of (111) is ^ 0, 
to obtain 



1 2x 2 ? 

(HO 7 l-y ^ x • 



we may square both sides 



( 112 ) 



Now since 1 - y ^ 0, we may multiply both sides of (112) without 
changing the inequality. Rearranging the resulting expression, we 
readily obtain 



y 2 * 



x 2 -l 



(113) 



Thus, 



tanh 



-1 



^ cosh ^ x 



implies that (113) holds. 



B. Proof of sufficiency: 

x 2 -l 

Assuming that y ^ ^2 and recalling that 0 £ y £ 1, we 



readily see that 



9 x 2 — 1 

V > T 2 * ~~x%~ 



(114) 



Writing this as 



r 2 > 1 - 



’ 



(115) 



a straightforward chain of manipulations leads us to 



(t-^ x 2 } k {x/x 2 -l}‘ 

''l-y J 



(116) 



Combining (114) and (115) , we see that 



1 ,2 



l-y 



- x z ;> 0, 



(117) 



so that we may extract square roots in (116) to obtain 



*t— X 2 ^ x/x 2 -l . 

l-y 



( 118 ) 
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Straightforward manipulations upon (118) lead us to 

^ x + /x z -l , 

/ 1-y 



(119) 



whence by (107) and (108) we obtain 



tanh 1 y ^ cosh ^ x« 



o x 2 -l 

Thus, y z ^ - 2 implies that (120) holds. 



( 120 ) 

Q.E.D. 



Let us now illustrate the use of Lemma 1. Consideration of (29) 
yields that T = T(C^) is given by 



T . -L- tanh' 1 

*6^ ( S 



( 121 ) 



s 

We also have that = t^(Ci-) is given by 



- — - — cosh ^ z. 



( 122 ) 



•V2 



Now C<- is reached from P° when T(C^) £ t^(C^) > since in both 
cases we use <j)*(t) = 0 for the final portion of the battle. Applying 
Lemma 1, we readily find that 



a l b l y 0 ^ Rs ( 1-1 / z2 ) • 



(123) 



Lemma 1 is also used to determine the condition 



a l b l y 0 ^ s2 + b (1>2 X 2^ 2 ’ 



(124) 



for D(cp . 
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11 , Discussion . 

For a discussion of the structure of the optimal allocation 
policies, the reader is directed to Section 6 of Appendix A. Here we 
will try to put into perspective what we have learned from this, our 
latest, examination of the Isbell-Marlow problem. 

First of all, we have concluded that the theory of SVIC is 
absolutely essential for solving such problems. Many of the results 
in the pioneering 1956 work of Isbell and Marlow [6] (see also [7]) 
are unsupported by such analysis (which uses theory developed since 
1956). When more than two target types are considered (i.e. n versus 
one combat) , the theory of SVIC must be used to determine the order 
in which it is optimal to annihilate target types. This was not 
essential for the simplest case of n = 2. 

Secondly, the algebraic method (see Section 4 of Appendix A) 
that we have developed for the synthesis of optimal policies appears 
to us as the only way to treat such problems. It should suffice for 
us to point out that Isbell and Marlow [6] (see also [7]) failed to 
discover the dispersal surfaces in the fire programming problem's 
solution when they used their geometric approach. Additionally, we 
do not feel that this geometric approach can be readily extended to 
problems with n target types (n > 2) , whereas our algebraic approach 
apparently can. 

Our re— examination of the Isbell-Marlow problem has been 

particularly fruitful, since it revealed the dominated payoff for 

S S 

0 £ <S < R - /R(R-l) , i.e. P° 6 D(C 5 ) fl D(C 5 ) implies that P 5 > P 5 - 
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We had previously observed such a phenomenon in the supporting weapon 
system game of H. K. Weiss [11] (see [10] and also Appendix B of [8]). 
At that time, we could not relate such a dominated payoff in a differ- 
ential game to any corresponding phenomenon in the simpler one-sided 
(optimal control) case. (It should be noted that Weiss did not use 
the subsequently well-known "in the small" necessary conditions of 
optimality from generalized control theory [4] (i.e. optimal control/ 
differential game theory) to develop his solution in [11] (it was 
erroneously reported that he did by S. Sternberg in [7]).) 

Finally, we would like to point out that our work here lays a 
firm foundation for considering two-sided problems. We propose this 
to ONR as a future research task. It should be noted that Sternberg's 
work [7] does not cite any of the essential theory of SVIC. In 
Appendix E we have pointed out that lack of consideration of the theory 
of SVIC is a gap in the current theory of differential games (particu- 
larly as applied to tactical allocation problems in the Lanchester 
theory of combat) . 
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Appendix G. The Prescribed Duration Battle. 



1. Introduction . 

One of the objectives of this research sponsored by ONR is to 
investigate the sensitivity of the optimal target selection policy to 
the form of the combat attrition model. Of particular interest is 
whether for the simplest target selection problem the structure of the 
optimal policy is sensitive to whether combat attrition is modelled 
as a deterministic or a stochastic process. In Appendix I we consider 
the optimal control of the Lanchester stochastic process for a pre- 
scribed duration battle between a homogeneous Y-force and a heteroge- 
neous enemy, and X^. In order to compare the structures of the 
optimal target selection policy for deterministic and stochastic attri- 
tion processes, it is, of course, necessary to develop a solution to 
the deterministic problem. We do that in this appendix. 

We have previously considered (several versions of) the pre- 
scribed duration battle [3], [4] (reproduced in this report as 
Appendices B and C) . Our subsequent examination here has led to a 
revision of one of our earlier tentative conclusions about the 
structure of the optimal target selection policy: we now conclude 
that when target types undergo a n square-law M attrition process 
(as we consider here) the optimal allocation may depend (indirectly) 
upon the force levels. We have reached this conclusion after a 
careful study of the problem at hand. 
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Also of interest as regards the sensitivity of the struc- 
ture of the optimal policy is whether the nature of the scenario 
(i.e. planning horizon) affects the structure. In [3] we compare 
the optimal target selection policy for the prescribed duration 
battle with that for a terminal control battle (a battle which 
only ends when force levels have reached some specified terminal 
state). After our re-examination here, we again will do this. 

In view of the extensive details on basic necessary condi- 
tions of optimality and synthesis of the optimal policy for simi- 
lar problems available elsewhere in this report (see, in particular, 
Appendices A, E, and F) , we will take the liberty of outlining some 
results here. As it is, our documentation of the problem’s solution 
is fairly lengthy. We may possibly present complete details in a 
future report. 

Finally, we note that a key aspect in our obtaining these 
new results has been our use of the theory of state variable in- 
equality constraints (SVIC) . The reader can find a fairly extensive 
discussion of this theory and its application to target selection 
problems in the Lanchester theory of combat in Appendix E. 
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2. The Optimal Control Problem. 



We consider the problem 



maximize {ry(T) - px^T) - qx 2 (T)} with T specified, 



subject to: = -‘t'a^y, 





( 1 ) 



x i ,x 2 *y s °» 



0 ^ 4 > «; 1 



and T <; T , 



where all symbols are defined in the next section. 

The battle lasts for 0 £ t <. unless, of course, one 

side or the other is annihilated before T^. To be more precise, 
the battle terminates under one of the following three conditions: 



where T denotes the time at which the battle ends. Upon further 
analysis, it has been convenient to consider that there are eight 
"terminal states," or "target sets." These are shown in Table I. 
The reader should note that for through Sg the battle ends 

when the force levels reach given terminal conditions. For these 



(1) x x (T) = x 2 (T) =0 and T £ ^ 



(2) y (T) = 0 and T <: T^ 



(3) T = T r 
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S 1 : x x (T) 

S 2 : X;l (T) 

S 3 : x 1 (T) 

S 4 : x i( T ) 

S 5 : x x (T) 

S 6 : Xl (T) 

S ? : Xl (T) 

S 8 : X 1 (T) 



Table I. Definition of Terminal States for 
Prescribed Duration Battle. 



> 0, x 2 (T) > 0, y (T) >0, T = T x 

= x 1 (t 1 ) = 0, Xz (T) > 0, y (T) >0, T = T 1 

where < T 

= x 1 (t 3 ) > 0, Xz (T) = 0, y (T) >0, T = T 1 

where t 3 < T 

> 0, Xz (T) > 0, y(T) = 0, T <L T 1 

= x 1 (t 1 ) = 0, Xz (T) > 0, y(T) =0, T ^ T 1 

where < T 

= Xl (t z ) > 0, Xz (T) = 0, y(T) =0, T ^ T x 

where t z < T 

= Xl (t x ) = 0, x z (T) = 0, y(T) >0, T ^ T x 

where < T 

= 0, x 2 (T) = x 2 (t 4 ) = 0 , y (T) > T ± 

where t, < T 
4 
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terminal states, T is undetermined, since it is determined by entry 
to the terminal state, and this depends upon the control used. For 
these cases, the well-known transversality condition must hold. 

3. Notation . 

The symbols which are used in this appendix are defined as 
follows : 

A = A(R,z) = [z 2 (R-1) - R]/(z-l) 2 , 

B = B(R,z) = A(z-1) 2 /z 2 = [z 2 (R~1) - R]/z 2 , 

= constant attrition-rate coefficients, 

D(S^) = domain of controllability for S^, 

p,q,r = utilities assigned to surviving X^, and Y 

forces respectively, 

p.(t) for i = 1,2,3 = dual variable corresponding to x^t) 

(x 3 (t) = y(t)), 

P for i = 1,..., 8 = payoff associated with extremal 
i 

leading to , 

P° = (x°,x°,y Q ) = point in the initial state space, 

R = a i b i/( a 2 b 2^ » 

A. k 

S for i = 1, . . . ,8 = the i— part of the terminal 
i 

surface as defined in Table I, 
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s = s(x°,x°) = b^x^ + b^x^ , 



t^ = time at which is annihilated, i.e. x^(t^) = 0, 

t^ = first time at which 2 b^x^(t 2 )x^ + b 2 (x 2 ) 2 = f° r 

an extremal leading to S, , 

o 



t^ = last time at which fire is directed at for an 

extremal leading to 



t^ = time at which is annihilated (before X^) , i.e. 

x (t.) = 0, for an extremal leading to S Q , 

2 4 o 



T = time at which battle ends. 



= Maximum possible duration for battle, i.e. T ^ T^, 



Tq = lower bound on (developed from condition x 2 ( T ) = 0) 
for an extremal leading to , 



V = v ( t ) = a 2 p 2 ( T ) - a^Or), 



x^,x 2> y = average force strengths; with initial values x°,x°,yQ, 
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2 U 2 l l yt V R-l * 



z = cosh \fk b t, (S. ) = 



a -r y\-L 

q V a ' 



6 = a 1 p/(a 2 q) , 



<J) = fraction of 



Y-fire directed at X^, 
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r = "backwards time" from the end of the battle defined by 
t = T-t, i.e. the time remaining before the end of the 
battle , 

?! (Sf) = "backwards time" of the first switch in tactics for 
extremals leading to S^, 



Additionally, remarks similar to those for above apply to 

t^S^, T (S ± ) , etc. 

4. Outline of Solution Procedure. 



Before giving our solution algoritm which is based upon 
Pontryagin f s maximum principle* [2], it seems appropriate to define 



some terms. We have then 



Definition 1: By an extremal path we mean a path 

on which the necessary conditions of 
optimality are everywhere satisfied 
(we use the word everywhere , since we 
assume that the class of admissible 
control functions is the class of 
piecewise-continuous functions) . 



Definition 2: By extremal control we mean the con- 

trol used in order that the system 
follow an extremal path. 

Definition 3: By the domain of controllability of a 

given terminal state we mean that sub- 
set of the initial state space from 
which extremals lead to the terminal 
state . 



* 

We use the form commonly used in this country. There is a sign 
difference between developments in the United States and those in the 
Soviet Union (see p. 108 of [1]). 
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Definition 4: By the synthesis of the optimal 

control , we mean the explicit deter- 
mination of the time history of the 
optimal control from initial to ter- 
minal time (possibly as a function of 
initial conditions) . 

Our solution algorithm then is as follows: 

(a) extremal control is determined by maximizing the Hamiltonian; 
since the state variables (force strengths) are non-negative, 
the control depends, in many cases, only on relationships 
between the dual variables (marginal return from destroying 
targets) , 

(b) from each separate terminal state, the time history of the 
dual variables is obtained by a backward integration of the 
adjoint system of differential equations combined with the 
extremal control; for a square law attrition process, the 
adjoint equations are independent of the state variables; 
the corner conditions must be considered in this step, 

(c) for each terminal state the domain of controllability is 
determined by forward integration of the state equations 
using the time history of extremal control developed in 
(b) ; changes in control with time (existence of transition 
surface) may have to be considered in this step. 

(d) the solution is determined at this point for regions of 
the initial state space which are covered by part of the 
domain of controllability for only one terminal state; 
one must also verify that the entire initial state space 
has been accounted for, since otherwise one may have over- 
looked some type of "singular" surface or even a terminal 
state , 

(e) if domains of controllability overlap so that for a point 
of £he initial state space contained in their intersection 
there is more than one extremal leading to the terminal 
surface, then one computes the payoff associated with each 
extremal; the optimal trajectory is selected from the ex- 
tremals by comparing these values. 

The above solution algorithm is essentially the same as that 
presented in Section 4 of Appendix A. Let us make a few remarks 
about the application of this procedure to the prescribed duration 
battle. In this case, we may think of time as being an additional 
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state variable. On the other hand, in the Isbell-Marlow terminal 
control problem, time may be considered to be a parameter and con- 
sequently was eliminated from the determinations in step (c) above. 

In other words, in the Isbell-Marlow problem a domain of controlla- 
bility was determined by inequalities involving the three state 
variables; in the prescribed duration battle such a determination 
as in step (c) above involves the four variables T, x°, x°, and y^ . 
For the prescribed duration battle we have not obtained (relatively) 
simple analytic expressions in step (c) of the above procedure. 
Consequently, we could not use analytic methods to accomplish steps 
(d) and (e). In other words, we have not been able to analytically 
determine when the domains of controllability corresponding to dif- 
ferent terminal states "overlap ." Hence, in many cases we have ana- 
lytically only been able to determine in extremal control. 

However, we can use computational methods to determine the 
optimal control. We have expressed our ‘solution" to this problem 
(presented in the next section) so that we can readily to the follow- 
ing: given a point P° = (x°, x^> yg) in the initial state space 

and T 1 we can determine which terminal states are reached by ex- 
1 f 

tremals. In other words, we can determine to which terminal states 
are reached by extremals. In other words, we can determine to 
which domains of controllability P belongs. Then, using the 
extremal control, we can compute numerically the payoff associated 
with each extremal and select the optimal policy from a finite 
number of possibilities. A M.S. thesis student (Robert L. Powers, 
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Lt., USN) is developing a digital computer program to use the 
M solution M of Tables II and III to determine the optimal target 
selection policy. Using this computer program, we plan to examine 
how much difference there is between payoffs from using optimal 
and non-optimal policies. 

5 . Summary of Solution. 

We have applied our solution procedure of Section 4 to 
develop a "solution" in the sense discussed there. Without loss of 
generality, we may assume that a ^b^ > a 2^2’ ^ #e * R > 1. Then, 

there are two cases to be considered 

( 1 ) 6 ;> 1 , 

and (2) 0 £ 6 < 1, 



where 6 - a^p/Ca^q). 

For Case (1): 6 ^ 1, the solution is shown in Tables II. 

In these tables, the domain of controllability and optimal policy 
is given for each terminal state. Since the domains of controlla- 
bility don't overlap, the extremals are unique. Since it may be 
shown that a solution exists for this optimization problem, the 
unique extremal control is consequently the optimal control. More- 
over, the optimal policy may be expressed in a particularly simple 
form: always concentrate all fire on while x^ > 0. We have 

presented complete details on the domain of controllability for each 
terminal state and given all "event" time for two reasons: (1) to be 
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Table II. Solution to Prescribed 

Duration Battle for <5^1. 



Nonrestrictive assumption: R > 1, i.e. a^b^ > a 2^2 



Case (1): <5^1 where 5 = a^p/(a^q) 

S x : x 1 (T) > 0, x 2 (T) > 0, y(T) > 0, T = T 

Optimal Control: 4>*(t) = 1 for 0 ^ t ^ T 



Case 1. > s2 



b < 



Case 2. a 1 b 1 y Q 2 < ^ 




Subcase A. a.jb y 2 ^ s 2 ~ (b 2 x °) 2 



T, < 



1 



£n 



V2 * \/ a l b l y 0 2 " " 2 + ( b 2 X 2 ) 



s - y Q 

2 < c 2 _ /k °\2 



Subcase B. < s z - ^ b 2 x ;?) 



Tl < 1 tantT 1 1 






Case 3. a b y 2 = s 2 
1 1 0 



T. < 



1 



£n 



o 

b 2 X 2 
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Table II. Solution to Prescribed 

Duration Battle for 6^1. 
(cont.) - 1 

S 2 : X;l (T) = x 1 (t 1 ) = 0, x 2 (T) > 0, y (T) >0, T = ^ 

where t^ < T 



Optimal Control: <J>*(t) = 



1 for 0 ^ t ^ t. 



0 for t 1 < t <; T 



where x^(t^) 



= 0 



9 u 

Always must have a^b^y^ > s2 ~ ^ b 2 X 2^ 2 
Case 1. a.^^ 2 ^ s2 + ( R- l)( b 2 x 2 ^ 2 



t, £ T„ < t„ + 



11 1 / — 7~‘ 

v a 2 b 2 



tanh 



-1 



b 2 x 2 vTT 



V a l b iyQ 2 - s2 + ( b 2 X 2 ) 2 



Case 2. a i b i y o 2 ^ s + ( R_1 )( b 2 x °) : 



t. <. T < t, + 



11 1 / — r— ■ 

v a 2 b 2 



tanh 



-1 



,v/a l b l y 0 2 - s 2 + (t» 2 x 2^ 2 



b 2 X 2 ^ 



where is given by 



(1) for a^b^yg 2 > s 2 






t = 






£n 



iVo 2 - s2 + <b 2*2 > 2 - b 



^3" \ 9 , o 

2 X 2 



^a, b ' y - s 
11^0 



(2) for a^b^y^2 < s : 
1 



, O 
b 2 X 2 






in 






_ / 



- s 2 + (b 2 x 2 > 2 ' 



s - y 0 
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Table II. Solution to Prescribed 

Duration Battle for 6^1. 
(cont.) - 2 

3^ 1 (concluded) 

(3) for a 1 b 1 y Q 2 = s2 

1 ( s 

«i - -=>■* — 

/a^b^ | ^2 X 2 



S 3 : Xl (T) = x x (t 3 ) > 0, x 2 (T) = 0, y (T) > 0, T = ^ 

where t 3 < T 



NO EXTREMALS LEAD TO THIS END STATE 



S 4 : x 1 (T) > 0, x 2 (T) > 0, y (T) = 0, T s ^ 

ptimal Control: <f) * (t ) = 1 for 0 ^ t £ T 



omain of Controllability: 



a iVo 



2 < s 2 - (b 2 x°) 2 



T = 






tanh 



-1 



y o 



£ T, 
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Table II. Solution to Prescribed 

Duration Battle for <5 ^ 1. 
(cont.) -3 



S 5 : X;l (T) = x 1 (t 1 ) = 0, x 2 (T) > 0, y (T) = 0, T £ T 



where t^ < T 



Optimal Control: 4>*(t) = 



1 for 0 £ t ^ t.^ where x^t^) = 0 



0 for t^ < t £ T 



Domain of Controllability: a^b^y^ 2 £ s 2 “ (b^x^) 2 



a l b l y 0 2 < ® 2 + (R-1 )( b 2 x 2 )2 



T = t 1 + 



1 ) /a^y ^ - s* + (b 2 xjp" 

tanh \ } ^ T, 






b 2 X 2° y/ ^ _ 



where is given by 

(1) for a^b^y^ 2 > s 2 



/a l b l y 0 2 ‘ S " + (b 2 X 2> Z " b 2 X 2 



C 1 " 






in 



XT y o _s 



(2) for a^b^ 2 < s 2 



b 2 x° - /a^y^ - s 2 + (b 2 ^>: 



C 1 = 



/a l b l 



in 



s - /a 1 b 1 y 0 



(3) for a^b^ 2 = s2 



c, *„ !— 

‘SA ( b 2 x 2 



Table II. Solution to Prescribed 



Duration Battle for ^ 1. 
(concluded) -4 

V X 1 (T) = X l (t 2 ) 51 °» X 2 (T) = °> y(T) " °> T * T 

where < T 

NO EXTREMALS LEAD TO THIS END STATE 

Sy : x 1 (T) = x^t.^) = 0, x 2 (T) = 0, y(T) >0, T ^ T^ 

where t^ < T 

( 1 for 0 ^ t £ t, where x 1 (t 1 ) = 
Optimal Control: <f>*(t) = c 

( 0 for < t £ T 

Domain of Controllability: a^b^y^ 2 > s 2 + (R-l)(b 2 x°)2 



T = t, + 

1 /alb 



tanh 



-1 



b 2 X 2 ^ 



2 2 



_L 

/a l b l y 0 Z ■ s " + (b 2 X 2> 2 j 1 



where t^ is given by 



/a^b^y^ 2 - s 2 + (b 0 x^ ) 2 - b„x,° 



2 2 ' 2 2 



b ‘ 



■Vi 



in 



) 



y o ■ s 



S g : X;l (T) = 0, x 2 (T) = x 2 (t 4 ) = 0, y(T) >0, T £ ^ 

where t. < T 
4 

NO EXTREMALS LEAD TO THIS END STATE 



able to make numerical comparisons with the stochastic control prob- 
lem considered in Appendix I, and (2) to be able to perform a sensi- 
tivity analysis on target selection policies. 

For Case (2): 0^5 < 1, the solution is shown in Tables III. 
It is expressed in the same format as for Case (1) with the exception 
that the controls are extremal (with two exceptions noted below) . 

It should be recalled that for the Isbell-Marlow terminal con- 
trol problem (see Appendix F) when 0 £ 6 < R -VR(R-l) some domains 
of controllability overlapped. Hence, extremals were not unique and 
considerations "in the large" had to be used to determine the optimal 
policy. We suspect that similar behavior happens in the prescribed 
duration battle, since we may consider the Isbell-Marlow problem to 
be imbedded in the prescribed duration battle (i.e. when T < T^, 
the solution to the Isbell-Marlow problem is applicable). Moreover, 
lack of explicit analytic expressions for two domains of controlla- 
bility (that for when there is a switch in tactics and S^) 

has prevented explicit determination as to which overlap. Further- 
more, the computation of the payoff associated with each (overlap- 
ping) extremal has not been analytically tractable. Hence, (except 
for two instances) extremal controls and (either implicit or ex- 
plicit) expressions for the corresponding domains of controllabil- 
ity are given in Tables III. However, as discussed in Section 4 
above, one can use the information presented in Tables III to 
numerically determine an optimal target selection policy for any 
specific set of input values. 
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Table III. Solution to Prescribed 

Duration Battle for 0 £ 5 < 1. 

Nonrestrictive assumption: R > 1, i.e. a.^ > a 2°2 

Case (2): 0 ^ <5 < 1 where 6 = a^p/Ca^q) 



S 1 : x^T) > 0, x 2 (T) > 0, y (T) > 0, T = T 1 



Since y(T) > 0, the switching time x^ = x^(S^) is given by 



T 1 = 



where z = 






in 



+ /z Z + a z - 1 



1 + a 



R - 6 , r 

and a = — , 



R - 1 




Case 1. x 1 ;> T 



Extremal Control: ij>*(t) = 0 for 0 <i t £ T 

Subcase A. a^b^y^ 2 < ^ | s2 “ ^^ x °) 2 

1 /a 1 b 1 ^ 

1 1 ^0 
' 

si^R - " 

Subcase B. a^b^y^ 2 s R | s 2 - (b^x°)2 

^ ^ o 



(1) for a^b^Q 2 > Rs" 



■ /a 1 b 1 y Q z - R{s z - (b^^)^} 



T 1 < 



•^Fi 



in 



- b 1 x ] 



* / a 1 b 1 y Q - s /r~ 



° /R~l 



G 



Table III. Solution to Prescribed 

Duration Battle for 0 ^ <5 < 1. 
(cont. )-l 



S. : (continued) 



(2) for < Rs2 



T i < 









b x° /R - /a b y A 2 - R{s 2 - (b^x^) 2 ) 
11 i i 0 1 r 



s / r ~ - /a 1 b 1 y Q 



(3) for a 1 b 1 y Q 2 = R s 2 



T 1 < 






£n 



b 2 x 2 



Case 2. x^ < T 



Extremal Control: <j>*(t) = 



1 for 0 ^ t £ T -t. 



0 for T - x^ < t ^ T 



Require that b^x^(T - x^) > 0, t>2 x 2^ > 0, y(T) > 0. 
where 



b^x^(T-x) = s cosh/a-^b^ (T-x^) - /a^b^ y^ sinh/a^b^ (T-x^) - ^2X2: 



b 2 X 2 <T) ={s cosh/a^b^ (T-x^) - / a^b^y^ sinh/a^b^CT-x^) } (coshv'a^b^ t^-I) 



-/a^bj{yp cosh/a^b^ (T-x^) sinh/a^b^(T-x^) Isinh/a^b^ x^ + b 2 X 2 ' 



^ 1^1 
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Table III. Solution to Prescribed 



S^: (concluded) 



Duration Battle for 0^6 < 1. 
(cont. )-2 



y(T) ={y Q cosh a^d-^) - — — sinh v^b7(T- T ^} cosh /Tb~ T 

/a^b^ 111 



s cosh /a 1 b 1 (T-Tj) - /a 1 b 1 y Q sinh /a 1 b 1 (T-t^ ^ inh/ r 2_ T 1 



*?2 



*> 2 • X ^(T) - x^(t^) 



Extremal Control: <J>*(t) = 



0, x 2 (T) > 0, y(T) >0, T = T 

where t^ < T 

1 for 0 £ t £ ^ where x^t ) = 0 



0 for t^ < t s T 



Always must have a^b^y^ 2 > s 2 - ( b 2 x 2^ 2 



C 1 + T 1 * T 1 



Case 1. a^b^ 2 ;> s 2 + (R-l)(b 2 x°) 2 



T 1 < ti + 






tanh 



-1 



b 2 x 2 °v^ 



Case 2, 



l/ a l b i y 0 2 - si +(b 2 x °)2 

a^y^ < s 2 + (R-l)(b 2 x°) 2 



T < t + 

1 v'aTb 



tanh 



-1 



l/a i b iy 0 2 “ s 2 +( b 2 X 2) : 



2 2 



b 2 X 2 ^ 



where = t^(S^) is given by 



z + y/zJ + a x - 1 



T 1 = 



^?2 



l n 



1 + a 



G 



Table III. Solution to Prescribed 



V 



(concluded) 
and z 



Duration Battle for 0 ^ <5 < 1. 
(cont • )-3 



- Rli _ r A 2 

R-l ’ “ q/ a„ ’ 



and where t^ = t^CS^) is given by 
(1) for a 1 ^ 1 y Q 2 > s 2 



h = 






Jin 



sZ + “ b 2 X i 



y 0 " s 



(2) for a^b Vq 2 < s 2 



t, = 



/S ih 



£n 



b 2 X 2 ' /a l b l y 0 Z ' sZ + (b 2 x 2 )1r 



s - y o 



(3) for 



1 1 J 0 



2 2 
* = 



h - 






£n 



Vi 



S 3 : x 1 (T) = x 1 (t 3 ) > 0, x 2 (T) = 0, y(T) >0, T = T ] 



Extremal Control: tf>*(t) = 



where t 3 < T 

1 for 0 ^ t ^ t 3 where explicit expression 

for t 3 is given below. 

0 for t 3 < t <i T 



Always must have a^b^y 2 ^ R {s 2 - (b^x°) 2 } 



t 3 ^ (see below) 



Case 1. a i l3 1 yQ 2 < s 2 + (R-l)(b 2 x°) 2 



VV T <V 
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Table III. Solution to Prescribed 



Duration Battle for 0 ^ 6 < 1 
(cont . )-4 



(continued) 



where is given below 



T(S 6 ) = t 2 (s 6 ) +-^= osh 



-l 



Rb 2 X 2 + v4 2 + R(R-l)(b 2 x°) 2 “ a 1 b 1 y Q 2 



(R-l)b 2 x 2 Ws 2 + R(R-l)(b 2 x°)^ -a^y^ 



y 2(t 2 ) = y 2 (t 2 (S 6 }) = Z 



(2R-1) b 2 x° + 2 '/s 2 + R(R-l) (b 2 x°) 2- a^b^y^ 2 



and where t 0 = t 0 (S ) is given by 
l lb 



(1) for a 1 b 1 y Q 2 > s 2 



t2<S6) ~ VaTb 



Jin 



11 



y(t 2 ) ~v/y z (t 2 ) - y Q 2 + s 2 /(a 1 b 1 ) 



y Q - 



(2) for a 1 b 1 y Q 2 < s 2 



t 2 (S 6 > = 



'Yi 



in 



v / 2 (t 2 ) “ y Q 2 + s 2 /(a 1 b 1 ) - y(t 2 ) 



s V 5 ?! ~ y 0 



(3) for a^y^ = s2 



w = 



Jin 



Ti 



y(t 2 ) 



Case 2. a^t^y^ 2 £ s2 + (R-l)(b 2 x°) 2 



Tq S T 1 < T (S ? ) 

where is given below 



T(S ? )= t 1 (S ? ) + 






tanh 



-1 



b 2 X 2° /F 



/a ] b 1 y 0 2 -s 2 + (b 2 x 2 > 2 , 



